First working iteration: check -> fetch -> verify for a real package

Rust PoC (cargo, packages.d/*.toml config) that checks astral-sh/uv's
GitHub Atom feed for a new release, fetches the matching asset via the
GitHub API, and verifies it. Confirmed live against the real repo: uv
actually ships GitHub build-provenance attestations (sigstore bundle) on
every release, so it's a tier-2 package, not the tier-4 same-origin-sha256
guessed in the original spec draft. Verified via `gh attestation verify`
rather than reimplementing sigstore in Rust. State persists across runs so
a second run correctly reports "up to date."

Also folds the finding back into SPEC.md: updates the uv example to
tier 2, derives tier from verification method instead of storing both
(avoids a tier/method mismatch that would mean nothing), marks the
packages.d/ layout question resolved, and updates Architecture/Status to
say what's actually implemented vs. still sketch (build/publish/review
queue, same-origin-sha256 against a live repo, scheduling, non-GitHub
sources, minisign are all still open).

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01A2FEut5tVMNjeVjqhgVZbr
This commit is contained in:
Austin Schaefer 2026-09-11 10:04:48 +02:00
parent 6361e9eb5e
commit bf46cbf073
11 changed files with 2145 additions and 24 deletions

3
.gitignore vendored Normal file
View file

@ -0,0 +1,3 @@
/target
/work
/state

1732
Cargo.lock generated Normal file

File diff suppressed because it is too large Load diff

13
Cargo.toml Normal file
View file

@ -0,0 +1,13 @@
[package]
name = "pkgwatch"
version = "0.1.0"
edition = "2024"
[dependencies]
anyhow = "1.0.104"
hex = "0.4.3"
regex = "1.13.1"
reqwest = { version = "0.13.5", default-features = false, features = ["blocking", "json", "rustls"] }
serde = { version = "1.0.229", features = ["derive"] }
sha2 = "0.11.0"
toml = "1.1.6"

99
SPEC.md
View file

@ -214,10 +214,15 @@ asset_pattern = "uv-x86_64-unknown-linux-gnu.tar.gz"
check_method = "github-atom" # or "github-api"; see Scaling > Check method
check_interval = "5m" # per-package, not a global daemon interval
# Verified 2026-09-11 against the real repo: uv publishes GitHub
# build-provenance attestations (sigstore bundle) for every release asset
# — tier 2, not the tier-4 same-origin-sha256 originally guessed here.
# Checked via `gh attestation verify` rather than reimplementing sigstore
# verification in Rust. Tier is *derived* from `method`, not stored
# separately — the PoC found that storing both invites a tier/method
# mismatch that would mean nothing (see `Verification::tier()` in the PoC).
[package.uv.verification]
tier = 4
method = "same-origin-sha256"
checksum_asset_pattern = "uv-x86_64-unknown-linux-gnu.tar.gz.sha256"
method = "github-attestation"
# Post-build sanity check — correctness only, not a security control.
# Runs the built binary and confirms it reports the version pkgwatch
@ -226,18 +231,42 @@ checksum_asset_pattern = "uv-x86_64-unknown-linux-gnu.tar.gz.sha256"
command = "uv --version"
version_regex = 'uv (\d+\.\d+\.\d+)'
# Tier 1 example:
# Tier 4 example — same-origin checksum only, proves transport integrity,
# not authorship (this is what the uv example above was, until checked):
[package.otherpkg]
source = "github-release"
repo = "someorg/otherpkg"
asset_pattern = "otherpkg-x86_64-unknown-linux-gnu.tar.gz"
[package.otherpkg.verification]
method = "same-origin-sha256"
checksum_asset_pattern = "otherpkg-x86_64-unknown-linux-gnu.tar.gz.sha256"
# Tier 1 example — not yet implemented in the PoC (only
# same-origin-sha256 and github-attestation exist so far):
[package.somepkg]
source = "url-with-version-regex"
url = "https://example.com/downloads/"
version_regex = 'somepkg-(\d+\.\d+\.\d+)\.tar\.gz'
[package.somepkg.verification]
tier = 1
method = "minisign"
pinned_key = "RWQ...base64pubkey..."
```
**PoC status** (see `src/`, `packages.d/uv.toml`): implements `repo`,
`asset_pattern`, and `verification.method` (`same-origin-sha256` |
`github-attestation` only), loaded from `packages.d/*.toml`. Confirmed
working end to end against the real `astral-sh/uv` repo — checks the
`github-atom` feed, fetches the matching release asset, verifies it via
`gh attestation verify`, and persists state so a second run reports
"up to date" instead of re-fetching. `source`, `check_method`,
`check_interval`, and `sanity_check` are still schema sketch, not yet read
by the code — the PoC only knows how to check GitHub-release sources.
Build/publish/review-queue (`makepkg`, `repo-add`, tier 46 human review)
are not implemented yet; a tier 46 pass currently just logs "flagging for
review" and stops.
Open questions on the schema:
- How much of `nvchecker`'s source-type taxonomy (github, gitlab, pypi,
@ -252,27 +281,37 @@ Open questions on the schema:
- Failure/alerting channel for tier 46 change events — log only, or a
notification hook (this box already has a wofi/Mako notification setup —
see `project_wofi_notification_picker` in Claude's memory).
- Config layout: single TOML vs. `packages.d/*.toml` directory (see
Scaling, above) — probably directory-based from the start, since
retrofitting later means a migration step for no benefit.
- ~~Config layout: single TOML vs. `packages.d/*.toml` directory~~
resolved: PoC loads `packages.d/*.toml` directly.
- Digest/batching rules for low-signal tier 46 changes (see Scaling,
above) — what counts as "low-signal" needs a concrete definition, not
just "not a major version bump."
## Architecture sketch
- **Config loader**: parses the TOML above into an in-memory package list.
- **Config loader**: parses `packages.d/*.toml` into an in-memory package
list. *(Implemented — `src/config.rs`.)*
- **Checker**: per source type, resolves "what's the latest version" —
likely reuses `nvchecker`'s logic/sources conceptually, possibly shells
out to it initially for the PoC rather than reimplementing every source
type in Rust. For GitHub sources, prefers the `github-atom` feed or
conditional-GET `github-api` calls (see Scaling > Check method) over
plain unconditional REST polling. Respects each package's own
`check_interval` rather than a single daemon-wide tick.
likely reuses `nvchecker`'s logic/sources conceptually for non-GitHub
sources eventually. For GitHub sources, prefers the `github-atom` feed
(see Scaling > Check method) over unconditional REST polling.
*(Implemented for GitHub only — `src/checker.rs` regex-matches the first
`releases/tag/<tag>` link in the feed rather than doing a full XML parse;
fine while the feed's newest-entry-first shape holds, revisit if that
ever changes. `check_interval`/per-package cadence not wired up yet —
the PoC is a single one-shot run, not a scheduled loop.)*
- **Fetcher**: downloads the artifact (and any checksum/signature/
attestation companion) for a resolved version.
- **Verifier**: tier-specific verification implementations behind a common
trait; returns a tier + pass/fail + justification string.
attestation companion) for a resolved version. *(Implemented —
`src/fetcher.rs`, via the GitHub releases API; exact asset-name match,
not a glob.)*
- **Verifier**: tier-specific verification implementations, dispatched via
a `Verification` enum matched on `method` (an internally-tagged serde
enum) rather than a trait — simpler while there are only two methods;
revisit as a trait if the method count grows. Returns a tier + pass/fail
+ justification string; tier is derived from `method`, never configured
separately. *(Implemented for `same-origin-sha256` and
`github-attestation``src/verifier.rs`. The latter shells out to `gh
attestation verify` rather than reimplementing sigstore verification.)*
- **Builder**: for tiers 13 on pass, generates/updates the PKGBUILD
(strict validation on any upstream-controlled string — version, filename
— before it touches generated shell content; never unescaped
@ -316,15 +355,27 @@ Open questions on the schema:
rejected — no repo-admin access on upstreams, and a relay-based
alternative would require a public receiver this box's networking
posture deliberately avoids.
- [x] `packages.d/*.toml` config layout implemented (`src/config.rs`).
- [x] First PoC iteration, working end to end against the real
`astral-sh/uv` repo: `github-atom` check → GitHub-API fetch →
`github-attestation` (tier 2) verify via `gh attestation verify`
state persisted so re-runs report "up to date." Confirmed uv
actually ships attestations, correcting the spec's original tier-4
guess for it. Run: `cargo run` from the project root.
- [ ] Not yet implemented: build (PKGBUILD generation + `makepkg`),
publish (`repo-add`), reviewer queue for tier 46, `same-origin-sha256`
exercised against a real package (code exists, untested against a
live repo), scheduling/`check_interval`, non-GitHub sources,
`minisign`/tier-1 method.
- [ ] Refine config schema further (see open questions above), including
the `sanity_check` block per package and `packages.d/` layout.
- [ ] Decide version-check strategy: shell out to `nvchecker` vs. own
implementation, for the PoC.
the `sanity_check` block per package.
- [ ] Decide version-check strategy for non-GitHub sources: shell out to
`nvchecker` vs. own implementation.
- [ ] Implement PKGBUILD generation with strict upstream-string validation
from day one (see Builder, above) — cheap to do right up front,
expensive to retrofit.
- [ ] PoC scope: single tier-4 package (e.g. `uv`, ironically) end-to-end —
check, fetch, same-origin-checksum verify, flag-for-review, manual
approve, build, post-build version sanity check, local repo publish.
- [ ] Next PoC iteration: carry the verified `uv` artifact through
build → sanity-check → `repo-add` publish, closing the loop to an
actual local pacman repo `pacman -Syu` can pick up.
- [ ] Decide on project home: local-only for now, or push to
code.austinschaefer.com (Forgejo) once the spec settles.

12
packages.d/uv.toml Normal file
View file

@ -0,0 +1,12 @@
# Verified 2026-09-11: astral-sh/uv publishes GitHub build-provenance
# attestations (sigstore bundle, `gh attestation verify` passes) for every
# release asset — tier 2, not the tier-4 same-origin-sha256 originally
# assumed in SPEC.md's draft example. Per-asset .sha256 files also exist
# but aren't used here since the attestation is strictly stronger.
[package.uv]
repo = "astral-sh/uv"
asset_pattern = "uv-x86_64-unknown-linux-gnu.tar.gz"
[package.uv.verification]
method = "github-attestation"

20
src/checker.rs Normal file
View file

@ -0,0 +1,20 @@
use anyhow::{Result, bail};
use regex::Regex;
/// Resolves the latest release tag for `repo` via its public Atom feed.
///
/// Deliberately not a full XML parse: the feed's newest entry is always
/// first, and its `<link rel="alternate" .../releases/tag/<tag>"/>` is the
/// first such link in the document, so a single regex match is sufficient.
/// Revisit with a real XML parser if GitHub's feed shape ever changes.
pub fn latest_github_release(client: &reqwest::blocking::Client, repo: &str) -> Result<String> {
let url = format!("https://github.com/{repo}/releases.atom");
let body = client.get(&url).send()?.error_for_status()?.text()?;
let pattern = format!(r#"releases/tag/([^"]+)""#);
let re = Regex::new(&pattern)?;
match re.captures(&body) {
Some(caps) => Ok(caps[1].to_string()),
None => bail!("no release tag found in {url}"),
}
}

54
src/config.rs Normal file
View file

@ -0,0 +1,54 @@
use anyhow::{Context, Result};
use serde::Deserialize;
use std::collections::HashMap;
use std::path::Path;
#[derive(Debug, Deserialize)]
struct PackageFile {
package: HashMap<String, Package>,
}
#[derive(Debug, Deserialize, Clone)]
pub struct Package {
pub repo: String,
pub asset_pattern: String,
pub verification: Verification,
}
#[derive(Debug, Deserialize, Clone)]
#[serde(tag = "method", rename_all = "kebab-case")]
pub enum Verification {
/// Tier 4: proves transport integrity only, not authorship. See
/// SPEC.md > Verification trust tiers.
SameOriginSha256 { checksum_asset_pattern: String },
/// Tier 2: GitHub build-provenance attestation, verified via `gh
/// attestation verify` rather than reimplementing sigstore in Rust.
GithubAttestation,
}
impl Verification {
pub fn tier(&self) -> u8 {
match self {
Verification::SameOriginSha256 { .. } => 4,
Verification::GithubAttestation => 2,
}
}
}
/// Loads every `*.toml` file in `dir` (the `packages.d/` layout from
/// SPEC.md > Scaling to many packages), keyed by package name.
pub fn load_packages_dir(dir: &Path) -> Result<Vec<(String, Package)>> {
let mut out = Vec::new();
for entry in std::fs::read_dir(dir).with_context(|| format!("reading {}", dir.display()))? {
let path = entry?.path();
if path.extension().and_then(|e| e.to_str()) != Some("toml") {
continue;
}
let text = std::fs::read_to_string(&path)
.with_context(|| format!("reading {}", path.display()))?;
let file: PackageFile = toml::from_str(&text)
.with_context(|| format!("parsing {}", path.display()))?;
out.extend(file.package);
}
Ok(out)
}

48
src/fetcher.rs Normal file
View file

@ -0,0 +1,48 @@
use anyhow::{Context, Result};
use serde::Deserialize;
use std::path::{Path, PathBuf};
#[derive(Debug, Deserialize)]
struct Release {
assets: Vec<Asset>,
}
#[derive(Debug, Deserialize)]
struct Asset {
name: String,
browser_download_url: String,
}
/// Downloads the release asset named exactly `asset_name` for `repo`@`tag`
/// into `dest_dir`, returning the local path.
pub fn download_asset(
client: &reqwest::blocking::Client,
repo: &str,
tag: &str,
asset_name: &str,
dest_dir: &Path,
) -> Result<PathBuf> {
let api_url = format!("https://api.github.com/repos/{repo}/releases/tags/{tag}");
let release: Release = client
.get(&api_url)
.send()?
.error_for_status()
.with_context(|| format!("fetching release metadata from {api_url}"))?
.json()?;
let asset = release
.assets
.iter()
.find(|a| a.name == asset_name)
.with_context(|| format!("no asset named '{asset_name}' in {repo}@{tag}"))?;
std::fs::create_dir_all(dest_dir)?;
let dest_path = dest_dir.join(&asset.name);
let bytes = client
.get(&asset.browser_download_url)
.send()?
.error_for_status()?
.bytes()?;
std::fs::write(&dest_path, &bytes)?;
Ok(dest_path)
}

77
src/main.rs Normal file
View file

@ -0,0 +1,77 @@
mod checker;
mod config;
mod fetcher;
mod state;
mod verifier;
use anyhow::Result;
use std::path::Path;
/// First iteration: check -> fetch -> verify -> report, for whatever is
/// in packages.d/. No build/publish step yet (see SPEC.md > Status).
fn main() -> Result<()> {
let client = reqwest::blocking::Client::builder()
.user_agent("pkgwatch/0.1 (PoC; https://code.austinschaefer.com)")
.build()?;
let packages_dir = Path::new("packages.d");
let state_dir = Path::new("state");
let work_dir = Path::new("work");
let packages = config::load_packages_dir(packages_dir)?;
if packages.is_empty() {
println!("no packages configured under {}/", packages_dir.display());
return Ok(());
}
for (name, pkg) in packages {
println!("== {name} ({}) ==", pkg.repo);
let latest = checker::latest_github_release(&client, &pkg.repo)?;
let last_seen = state::load_last_version(state_dir, &name);
if last_seen.as_deref() == Some(latest.as_str()) {
println!(" up to date at {latest}");
continue;
}
println!(" new version detected: {latest} (previously: {last_seen:?})");
let dest_dir = work_dir.join(&name).join(&latest);
let artifact_path =
fetcher::download_asset(&client, &pkg.repo, &latest, &pkg.asset_pattern, &dest_dir)?;
println!(" fetched {}", artifact_path.display());
let result = verifier::verify(
&client,
&pkg.verification,
&pkg.repo,
&latest,
&artifact_path,
&dest_dir,
)?;
println!(
" verification (tier {}): {} — {}",
result.tier,
if result.passed { "PASS" } else { "FAIL" },
result.justification
);
match (result.tier, result.passed) {
(1..=3, true) => {
println!(" tier 1-3 pass: would auto-build + publish (not yet implemented)");
state::save_last_version(state_dir, &name, &latest)?;
}
(_, true) => {
println!(" tier 4-6 pass: flagging for human review, not auto-publishing");
println!(" (review-queue persistence not yet implemented — this is where it plugs in)");
}
(_, false) => {
println!(" verification failed — not publishing, not updating state");
}
}
}
Ok(())
}

18
src/state.rs Normal file
View file

@ -0,0 +1,18 @@
use anyhow::Result;
use std::path::Path;
/// Last-known-published version per package, so re-runs don't re-flag a
/// version already handled. Deliberately just one file per package for
/// now — this is where a real review-queue persistence layer plugs in
/// later (see SPEC.md > Architecture > Reviewer queue).
pub fn load_last_version(state_dir: &Path, name: &str) -> Option<String> {
std::fs::read_to_string(state_dir.join(format!("{name}.version")))
.ok()
.map(|s| s.trim().to_string())
}
pub fn save_last_version(state_dir: &Path, name: &str, version: &str) -> Result<()> {
std::fs::create_dir_all(state_dir)?;
std::fs::write(state_dir.join(format!("{name}.version")), version)?;
Ok(())
}

93
src/verifier.rs Normal file
View file

@ -0,0 +1,93 @@
use crate::config::Verification;
use crate::fetcher;
use anyhow::{Context, Result};
use sha2::{Digest, Sha256};
use std::path::Path;
use std::process::Command;
pub struct VerificationResult {
pub tier: u8,
pub passed: bool,
pub justification: String,
}
/// Runs the verification method declared for a package against a
/// downloaded artifact. See SPEC.md > Verification trust tiers for what
/// each tier does and does not prove.
pub fn verify(
client: &reqwest::blocking::Client,
verification: &Verification,
repo: &str,
tag: &str,
artifact_path: &Path,
dest_dir: &Path,
) -> Result<VerificationResult> {
match verification {
Verification::SameOriginSha256 {
checksum_asset_pattern,
} => {
let checksum_path = fetcher::download_asset(
client,
repo,
tag,
checksum_asset_pattern,
dest_dir,
)?;
let checksum_text = std::fs::read_to_string(&checksum_path)?;
let expected = checksum_text
.split_whitespace()
.next()
.context("empty checksum file")?
.to_lowercase();
let data = std::fs::read(artifact_path)?;
let actual = sha256_hex(&data);
let passed = actual == expected;
Ok(VerificationResult {
tier: verification.tier(),
passed,
justification: if passed {
"same-origin sha256 matched — proves transport integrity only, \
not authorship (see tier 4 in SPEC.md)"
.into()
} else {
format!("sha256 mismatch: expected {expected}, got {actual}")
},
})
}
Verification::GithubAttestation => {
let output = Command::new("gh")
.args([
"attestation",
"verify",
&artifact_path.to_string_lossy(),
"-R",
repo,
])
.output()
.context("running `gh attestation verify` (is `gh` installed and authenticated?)")?;
let passed = output.status.success();
Ok(VerificationResult {
tier: verification.tier(),
passed,
justification: if passed {
"GitHub build-provenance attestation verified via `gh attestation verify`"
.into()
} else {
format!(
"gh attestation verify failed: {}",
String::from_utf8_lossy(&output.stderr).trim()
)
},
})
}
}
}
fn sha256_hex(data: &[u8]) -> String {
let mut hasher = Sha256::new();
hasher.update(data);
hex::encode(hasher.finalize())
}