Track scaleway-cli, add version-templated assets and combined checksums
Wire up scaleway/scaleway-cli as a watched package (extra's copy lags
weeks behind upstream). Getting a real second package running exposed
gaps uv's config never hit:
- asset_pattern/checksum_asset_pattern now support a {version}
placeholder for release assets that embed the version in the
filename (checker::version_from_tag strips a tag's leading `v`).
- same-origin-sha256 now matches the checksum line by filename instead
of assuming a single-hash file, to support combined multi-asset
checksum files like scaleway-cli's SHA256SUMS.
- latest_github_release now confirms each Atom-feed candidate against
the releases API instead of trusting the newest entry outright —
scaleway-cli publishes a `-dbg1` tag with no real Release object that
otherwise sorts newest in the feed.
Confirmed correct against the real repo: no attestations upstream, so
tier 4 (same-origin-sha256) applies, and the checker correctly skips
the dbg1 tag to land on the real latest release.
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
This commit is contained in:
parent
388096264b
commit
7f4bb4cccf
6 changed files with 152 additions and 27 deletions
53
SPEC.md
53
SPEC.md
|
|
@ -254,13 +254,35 @@ method = "minisign"
|
||||||
pinned_key = "RWQ...base64pubkey..."
|
pinned_key = "RWQ...base64pubkey..."
|
||||||
```
|
```
|
||||||
|
|
||||||
**PoC status** (see `src/`, `packages.d/uv.toml`): implements `repo`,
|
**PoC status** (see `src/`, `packages.d/uv.toml`, `packages.d/scaleway-cli.toml`):
|
||||||
`asset_pattern`, and `verification.method` (`same-origin-sha256` |
|
implements `repo`, `asset_pattern`, and `verification.method`
|
||||||
`github-attestation` only), loaded from `packages.d/*.toml`. Confirmed
|
(`same-origin-sha256` | `github-attestation` only), loaded from
|
||||||
working end to end against the real `astral-sh/uv` repo — checks the
|
`packages.d/*.toml`. Confirmed working end to end against two real repos:
|
||||||
`github-atom` feed, fetches the matching release asset, verifies it via
|
|
||||||
`gh attestation verify`, and persists state so a second run reports
|
- `astral-sh/uv` — `github-atom` feed → fetch → `gh attestation verify`
|
||||||
"up to date" instead of re-fetching. `source`, `check_method`,
|
(tier 2) → state persisted so a second run reports "up to date."
|
||||||
|
- `scaleway/scaleway-cli` — first live exercise of `same-origin-sha256`
|
||||||
|
(tier 4). Verified upstream ships no build-provenance attestations
|
||||||
|
(`attestations` API 404s), so this is genuinely tier 4, not an
|
||||||
|
under-verified tier 2. Surfaced two schema/implementation gaps beyond
|
||||||
|
what uv exercised, both now handled:
|
||||||
|
- Release asset names embed the version
|
||||||
|
(`scaleway-cli_2.62.0_linux_amd64`), unlike uv's static names.
|
||||||
|
`asset_pattern`/`checksum_asset_pattern` now support a `{version}`
|
||||||
|
placeholder, substituted via `checker::version_from_tag` (which also
|
||||||
|
strips a tag's leading `v`, since scaleway-cli tags `vX.Y.Z` but
|
||||||
|
filenames use the bare version).
|
||||||
|
- Checksums ship as one combined `SHA256SUMS` (one line per platform
|
||||||
|
asset) rather than a per-asset file like uv's — the verifier now
|
||||||
|
matches the line by filename instead of assuming a single-hash file.
|
||||||
|
- Separately, scaleway-cli's Atom feed lists a `vX.Y.Z-dbg1` tag newest,
|
||||||
|
with no real Release object behind it (`releases/tags/<tag>` 404s) —
|
||||||
|
`checker::latest_github_release` now confirms each feed candidate
|
||||||
|
against the releases API in feed order rather than trusting the first
|
||||||
|
entry outright.
|
||||||
|
|
||||||
|
Tier 4-6 packages (scaleway-cli included) are not auto-published — see
|
||||||
|
Build/publish/review-queue below. `source`, `check_method`,
|
||||||
`check_interval`, and `sanity_check` are still schema sketch, not yet read
|
`check_interval`, and `sanity_check` are still schema sketch, not yet read
|
||||||
by the code — the PoC only knows how to check GitHub-release sources.
|
by the code — the PoC only knows how to check GitHub-release sources.
|
||||||
Build/publish/review-queue (`makepkg`, `repo-add`, tier 4–6 human review)
|
Build/publish/review-queue (`makepkg`, `repo-add`, tier 4–6 human review)
|
||||||
|
|
@ -362,11 +384,20 @@ Open questions on the schema:
|
||||||
state persisted so re-runs report "up to date." Confirmed uv
|
state persisted so re-runs report "up to date." Confirmed uv
|
||||||
actually ships attestations, correcting the spec's original tier-4
|
actually ships attestations, correcting the spec's original tier-4
|
||||||
guess for it. Run: `cargo run` from the project root.
|
guess for it. Run: `cargo run` from the project root.
|
||||||
|
- [x] `same-origin-sha256` exercised against a real package:
|
||||||
|
`scaleway/scaleway-cli`, tracked via `packages.d/scaleway-cli.toml`
|
||||||
|
(added because Manjaro's `extra` scaleway-cli lags upstream). Tier 4
|
||||||
|
confirmed correct — no build-provenance attestations upstream.
|
||||||
|
Required adding `{version}`-placeholder support to `asset_pattern`/
|
||||||
|
`checksum_asset_pattern`, filename-matched parsing of combined
|
||||||
|
multi-asset checksum files, and having `latest_github_release`
|
||||||
|
confirm each Atom-feed candidate against the releases API (this
|
||||||
|
repo's newest feed entry, a `-dbg1` tag, has no real Release behind
|
||||||
|
it). Still just flags for human review, same as any tier 4-6 pass —
|
||||||
|
not auto-installed; see the unchecked build/publish item below.
|
||||||
- [ ] Not yet implemented: build (PKGBUILD generation + `makepkg`),
|
- [ ] Not yet implemented: build (PKGBUILD generation + `makepkg`),
|
||||||
publish (`repo-add`), reviewer queue for tier 4–6, `same-origin-sha256`
|
publish (`repo-add`), reviewer queue for tier 4–6, scheduling/
|
||||||
exercised against a real package (code exists, untested against a
|
`check_interval`, non-GitHub sources, `minisign`/tier-1 method.
|
||||||
live repo), scheduling/`check_interval`, non-GitHub sources,
|
|
||||||
`minisign`/tier-1 method.
|
|
||||||
- [ ] Refine config schema further (see open questions above), including
|
- [ ] Refine config schema further (see open questions above), including
|
||||||
the `sanity_check` block per package.
|
the `sanity_check` block per package.
|
||||||
- [ ] Decide version-check strategy for non-GitHub sources: shell out to
|
- [ ] Decide version-check strategy for non-GitHub sources: shell out to
|
||||||
|
|
|
||||||
17
packages.d/scaleway-cli.toml
Normal file
17
packages.d/scaleway-cli.toml
Normal file
|
|
@ -0,0 +1,17 @@
|
||||||
|
# Verified 2026-09-17 against the real repo: scaleway/scaleway-cli does not
|
||||||
|
# publish GitHub build-provenance attestations (the attestations API 404s
|
||||||
|
# for a fresh release asset) — tier 4, same-origin checksum only, not tier
|
||||||
|
# 2. Motivation: Manjaro's `extra` scaleway-cli lags upstream by weeks;
|
||||||
|
# this tracks upstream releases directly instead.
|
||||||
|
#
|
||||||
|
# Releases ship one combined `SHA256SUMS` file (one line per platform
|
||||||
|
# asset) rather than a per-asset checksum file like uv's — verifier
|
||||||
|
# matches the line by filename.
|
||||||
|
|
||||||
|
[package.scaleway-cli]
|
||||||
|
repo = "scaleway/scaleway-cli"
|
||||||
|
asset_pattern = "scaleway-cli_{version}_linux_amd64"
|
||||||
|
|
||||||
|
[package.scaleway-cli.verification]
|
||||||
|
method = "same-origin-sha256"
|
||||||
|
checksum_asset_pattern = "SHA256SUMS"
|
||||||
|
|
@ -3,17 +3,46 @@ use regex::Regex;
|
||||||
|
|
||||||
/// Resolves the latest release tag for `repo` via its public Atom feed.
|
/// Resolves the latest release tag for `repo` via its public Atom feed.
|
||||||
///
|
///
|
||||||
/// Deliberately not a full XML parse: the feed's newest entry is always
|
/// Deliberately not a full XML parse: the feed lists entries newest-first,
|
||||||
/// first, and its `<link rel="alternate" .../releases/tag/<tag>"/>` is the
|
/// and each `<link rel="alternate" .../releases/tag/<tag>"/>` is matched
|
||||||
/// first such link in the document, so a single regex match is sufficient.
|
/// in document order. Revisit with a real XML parser if GitHub's feed
|
||||||
/// Revisit with a real XML parser if GitHub's feed shape ever changes.
|
/// shape ever changes.
|
||||||
|
///
|
||||||
|
/// The feed can list a tag newer than any tag with a real Release object
|
||||||
|
/// behind it — observed on scaleway/scaleway-cli, which pushes a
|
||||||
|
/// `vX.Y.Z-dbg1` tag (no corresponding Release; `releases/tags/<tag>`
|
||||||
|
/// 404s) right after each real release, and that tag sorts newest in the
|
||||||
|
/// feed. So each candidate is confirmed against the releases API in feed
|
||||||
|
/// order, returning the first that actually resolves.
|
||||||
pub fn latest_github_release(client: &reqwest::blocking::Client, repo: &str) -> Result<String> {
|
pub fn latest_github_release(client: &reqwest::blocking::Client, repo: &str) -> Result<String> {
|
||||||
let url = format!("https://github.com/{repo}/releases.atom");
|
let url = format!("https://github.com/{repo}/releases.atom");
|
||||||
let body = client.get(&url).send()?.error_for_status()?.text()?;
|
let body = client.get(&url).send()?.error_for_status()?.text()?;
|
||||||
|
|
||||||
let re = Regex::new(r#"releases/tag/([^"]+)""#)?;
|
let re = Regex::new(r#"releases/tag/([^"]+)""#)?;
|
||||||
match re.captures(&body) {
|
let mut candidates = re
|
||||||
Some(caps) => Ok(caps[1].to_string()),
|
.captures_iter(&body)
|
||||||
None => bail!("no release tag found in {url}"),
|
.map(|caps| caps[1].to_string())
|
||||||
|
.peekable();
|
||||||
|
if candidates.peek().is_none() {
|
||||||
|
bail!("no release tag found in {url}");
|
||||||
}
|
}
|
||||||
|
|
||||||
|
for tag in candidates {
|
||||||
|
let release_url = format!("https://api.github.com/repos/{repo}/releases/tags/{tag}");
|
||||||
|
if client.get(&release_url).send()?.status().is_success() {
|
||||||
|
return Ok(tag);
|
||||||
|
}
|
||||||
|
}
|
||||||
|
bail!("no release tag in {url} resolved to a real release via the API")
|
||||||
|
}
|
||||||
|
|
||||||
|
/// Strips a leading `v` from a release tag, e.g. `v2.62.0` -> `2.62.0`.
|
||||||
|
///
|
||||||
|
/// Some projects (uv) tag releases with the bare version and use it
|
||||||
|
/// verbatim in asset filenames; others (scaleway-cli) tag `vX.Y.Z` but
|
||||||
|
/// still use the bare version in filenames. This is the version string
|
||||||
|
/// substituted into `{version}` placeholders in `asset_pattern` /
|
||||||
|
/// `checksum_asset_pattern`, not the tag used for API/attestation calls.
|
||||||
|
pub fn version_from_tag(tag: &str) -> &str {
|
||||||
|
tag.strip_prefix('v').unwrap_or(tag)
|
||||||
}
|
}
|
||||||
|
|
|
||||||
|
|
@ -11,6 +11,11 @@ struct PackageFile {
|
||||||
#[derive(Debug, Deserialize, Clone)]
|
#[derive(Debug, Deserialize, Clone)]
|
||||||
pub struct Package {
|
pub struct Package {
|
||||||
pub repo: String,
|
pub repo: String,
|
||||||
|
/// Exact GitHub release asset name (still not a glob — see
|
||||||
|
/// SPEC.md > Architecture > Fetcher), optionally containing a
|
||||||
|
/// `{version}` placeholder for projects whose asset names embed the
|
||||||
|
/// version (e.g. `scaleway-cli_{version}_linux_amd64`). Substituted via
|
||||||
|
/// `checker::version_from_tag` before matching.
|
||||||
pub asset_pattern: String,
|
pub asset_pattern: String,
|
||||||
pub verification: Verification,
|
pub verification: Verification,
|
||||||
}
|
}
|
||||||
|
|
@ -19,7 +24,8 @@ pub struct Package {
|
||||||
#[serde(tag = "method", rename_all = "kebab-case")]
|
#[serde(tag = "method", rename_all = "kebab-case")]
|
||||||
pub enum Verification {
|
pub enum Verification {
|
||||||
/// Tier 4: proves transport integrity only, not authorship. See
|
/// Tier 4: proves transport integrity only, not authorship. See
|
||||||
/// SPEC.md > Verification trust tiers.
|
/// SPEC.md > Verification trust tiers. `checksum_asset_pattern` may
|
||||||
|
/// also contain a `{version}` placeholder, same as `asset_pattern`.
|
||||||
SameOriginSha256 { checksum_asset_pattern: String },
|
SameOriginSha256 { checksum_asset_pattern: String },
|
||||||
/// Tier 2: GitHub build-provenance attestation, verified via `gh
|
/// Tier 2: GitHub build-provenance attestation, verified via `gh
|
||||||
/// attestation verify` rather than reimplementing sigstore in Rust.
|
/// attestation verify` rather than reimplementing sigstore in Rust.
|
||||||
|
|
|
||||||
|
|
@ -38,8 +38,11 @@ fn main() -> Result<()> {
|
||||||
println!(" new version detected: {latest} (previously: {last_seen:?})");
|
println!(" new version detected: {latest} (previously: {last_seen:?})");
|
||||||
|
|
||||||
let dest_dir = work_dir.join(&name).join(&latest);
|
let dest_dir = work_dir.join(&name).join(&latest);
|
||||||
|
let asset_name = pkg
|
||||||
|
.asset_pattern
|
||||||
|
.replace("{version}", checker::version_from_tag(&latest));
|
||||||
let artifact_path =
|
let artifact_path =
|
||||||
fetcher::download_asset(&client, &pkg.repo, &latest, &pkg.asset_pattern, &dest_dir)?;
|
fetcher::download_asset(&client, &pkg.repo, &latest, &asset_name, &dest_dir)?;
|
||||||
println!(" fetched {}", artifact_path.display());
|
println!(" fetched {}", artifact_path.display());
|
||||||
|
|
||||||
let result = verifier::verify(
|
let result = verifier::verify(
|
||||||
|
|
|
||||||
|
|
@ -1,6 +1,7 @@
|
||||||
|
use crate::checker::version_from_tag;
|
||||||
use crate::config::Verification;
|
use crate::config::Verification;
|
||||||
use crate::fetcher;
|
use crate::fetcher;
|
||||||
use anyhow::{Context, Result};
|
use anyhow::{Context, Result, bail};
|
||||||
use sha2::{Digest, Sha256};
|
use sha2::{Digest, Sha256};
|
||||||
use std::path::Path;
|
use std::path::Path;
|
||||||
use std::process::Command;
|
use std::process::Command;
|
||||||
|
|
@ -26,14 +27,16 @@ pub fn verify(
|
||||||
Verification::SameOriginSha256 {
|
Verification::SameOriginSha256 {
|
||||||
checksum_asset_pattern,
|
checksum_asset_pattern,
|
||||||
} => {
|
} => {
|
||||||
|
let checksum_asset_name =
|
||||||
|
checksum_asset_pattern.replace("{version}", version_from_tag(tag));
|
||||||
let checksum_path =
|
let checksum_path =
|
||||||
fetcher::download_asset(client, repo, tag, checksum_asset_pattern, dest_dir)?;
|
fetcher::download_asset(client, repo, tag, &checksum_asset_name, dest_dir)?;
|
||||||
let checksum_text = std::fs::read_to_string(&checksum_path)?;
|
let checksum_text = std::fs::read_to_string(&checksum_path)?;
|
||||||
let expected = checksum_text
|
let artifact_name = artifact_path
|
||||||
.split_whitespace()
|
.file_name()
|
||||||
.next()
|
.and_then(|n| n.to_str())
|
||||||
.context("empty checksum file")?
|
.context("artifact path has no filename")?;
|
||||||
.to_lowercase();
|
let expected = expected_checksum(&checksum_text, artifact_name)?;
|
||||||
|
|
||||||
let data = std::fs::read(artifact_path)?;
|
let data = std::fs::read(artifact_path)?;
|
||||||
let actual = sha256_hex(&data);
|
let actual = sha256_hex(&data);
|
||||||
|
|
@ -88,3 +91,39 @@ fn sha256_hex(data: &[u8]) -> String {
|
||||||
hasher.update(data);
|
hasher.update(data);
|
||||||
hex::encode(hasher.finalize())
|
hex::encode(hasher.finalize())
|
||||||
}
|
}
|
||||||
|
|
||||||
|
/// Finds the expected hash for `artifact_name` in a checksum file.
|
||||||
|
///
|
||||||
|
/// Handles both a bare-hash file covering a single asset (e.g. uv's
|
||||||
|
/// `<name>.sha256`) and a combined multi-asset file (e.g. scaleway-cli's
|
||||||
|
/// `SHA256SUMS`, one `<hash> <filename>` or `<hash> *<filename>` line per
|
||||||
|
/// platform asset) by matching on the filename field when one is present.
|
||||||
|
fn expected_checksum(checksum_text: &str, artifact_name: &str) -> Result<String> {
|
||||||
|
let lines: Vec<&str> = checksum_text
|
||||||
|
.lines()
|
||||||
|
.filter(|l| !l.trim().is_empty())
|
||||||
|
.collect();
|
||||||
|
|
||||||
|
for line in &lines {
|
||||||
|
let mut parts = line.split_whitespace();
|
||||||
|
let hash = parts.next().context("empty checksum line")?;
|
||||||
|
if let Some(name) = parts.next()
|
||||||
|
&& name.trim_start_matches('*') == artifact_name
|
||||||
|
{
|
||||||
|
return Ok(hash.to_lowercase());
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
if let [line] = lines.as_slice() {
|
||||||
|
let hash = line
|
||||||
|
.split_whitespace()
|
||||||
|
.next()
|
||||||
|
.context("empty checksum file")?;
|
||||||
|
return Ok(hash.to_lowercase());
|
||||||
|
}
|
||||||
|
|
||||||
|
bail!(
|
||||||
|
"no checksum entry for '{artifact_name}' found in checksum file ({} lines)",
|
||||||
|
lines.len()
|
||||||
|
)
|
||||||
|
}
|
||||||
|
|
|
||||||
Loading…
Reference in a new issue