pkgwatch/SPEC.md

# pkgwatch — declarative package-update watcher/publisher (working name)

Status: design draft, pre-PoC. Captures the design discussion as of 2026-09-11.

## Problem

Software not packaged by the distro (Arch/Manjaro here) usually gets installed
one of a few ways:

- `curl | sh` from the vendor's own install script — the classic "trust me,
  bro." The script and any checksum it embeds share a trust boundary, so it
  verifies nothing beyond transport corruption.
- Manual download + manual checksum/signature verification, redone by hand
  every time you want to update. Tedious enough that people stop doing it.
- A distro package (pacman `extra`, AUR) — trustworthy, but version-lagged
  behind upstream, and someone else has to maintain the PKGBUILD.

There's no local, low-effort way to say "here's how to fetch and verify
package X" once, and have that declaration stay live — checked periodically,
re-verified on every new upstream release, and fed into a normal
`pacman`-based workflow without hand-editing a PKGBUILD each time.

## Scope

This is a personal tool for a small, curated list of non-critical packages
— not a general-purpose supply-chain-security framework. The concrete
motivating case: software (especially fast-moving AI/ML tooling) that
Manjaro's `extra` repo lags weeks behind upstream on, where raw AUR or a
vendor's curl|sh script are the only faster alternatives today.

**Goal**: a middle ground — fresher than Manjaro's lag, with real
verification where the vendor actually offers something to check, safer
than blindly piping an install script to `sh`.

**Non-goals**:

- Defending against a fully compromised vendor signing/release pipeline.
  If upstream's CI or signing key is itself compromised, pkgwatch cannot
  and does not try to catch that. Tiers 1–3 (below) raise the bar from
  "trust the domain" to "trust the vendor's actual release process"; they
  are not a guarantee against that process being subverted.
- Defending against a malicious downgrade specifically. A compromised
  version-check source reporting an older version as "latest" is a
  downstream/distro-security problem, out of scope here. (See "post-build
  version check" below for a related but distinct sanity check — it is
  not a security control.)
- Supporting an adversarial or multi-user config. The tracked-package list
  is curated by the one person running the daemon on their own machine;
  config/state file integrity relies on normal filesystem permissions, not
  a hardened trust boundary. Key-pinning friction on rotation (see tiers
  below) is acceptable, even desirable, regardless of list size — it's
  about who's allowed to add a trust decision, not about volume.

This is a narrower non-goal than "doesn't need to scale" — see **Scaling to
many packages**, below. If pkgwatch actually solves the release-cadence
problem, the natural outcome is tracking many packages, not a handful, and
the design should hold up under that.

## Vision

A Rust binary, run as a systemd service (service + timer, periodic not
persistent), that:

1. Reads a declarative config of tracked packages — where to check for new
   versions, how to fetch the artifact, and how to verify it.
2. On each run, checks each tracked package for a new upstream version.
3. If a new version is found, fetches the artifact and runs the verification
   method declared for that package.
4. If verification succeeds (per the package's trust tier — see below),
   updates/generates a local PKGBUILD (bump `pkgver`, refresh
   `sha256sums`/signature reference) and rebuilds it into a local pacman
   repo via `makepkg` + `repo-add`.
5. The next `pacman -Syu` (with the local repo configured) picks up the new
   version normally — no separate tooling needed on the consuming side.

This is conceptually `nvchecker` (version checking) + `updpkgsums` (checksum
refresh) + `repo-add` (local repo publishing), fused into one daemon with a
single declarative source of truth, plus an explicit, surfaced trust model
that those tools don't provide.

## Verification trust tiers

Per the Scope above, the goal is to raise the bar above curl|sh where the
vendor gives us something to check — not to build airtight supply-chain
defense. The central design problem within that goal: from the install UX,
a cryptographically strong verification and a "trust-me-bro" same-domain
checksum look identical. The tool's job is to make that difference legible
instead of laundering every package into an undifferentiated "verified"
bucket.

Tiers, strongest to weakest:

1. **Pinned-key signature** — GPG/minisign/sigstore-cosign, where the
   signing key's fingerprint is pinned in *our* config (not fetched fresh
   from the vendor each time). Proves authorship, independent of the
   artifact's own hosting.
2. **Build provenance attestation** — GitHub/GitLab attestations, SLSA
   provenance. Ties the artifact to a specific CI run and source commit.
   Strong, but only as trustworthy as that CI pipeline.
3. **Registry-native signing** — crates.io, PyPI trusted publishing, npm
   provenance. Similar strength, scoped to that registry's trust model.
4. **Same-origin checksum file** — a `.sha256`/`.sha256sum` served next to
   the artifact by the vendor. Proves transport integrity only. If the
   vendor's server or account is compromised, the attacker controls the
   artifact and the "verification" in the same move.
5. **Checksum embedded in an install script** — the classic curl|sh case.
   The verifier and the thing being verified share a trust boundary.
6. **Nothing** — bare TLS to a domain, no checksum or signature at all.

### Automation posture per tier

This is the load-bearing decision, not just a cosmetic label:

- **Tiers 1–3**: a passing verification is a real trust signal. Safe to
  auto-bump, auto-verify, auto-publish unattended.
- **Tiers 4–6**: a passing "verification" only proves internal consistency
  of one origin (the checksum and the artifact agree), which tells you
  nothing about whether that origin was compromised. For these tiers the
  tool should **not** treat a pass as "verified, ship it." Instead: treat a
  version/hash *change* as a flag-for-human-review event. The value pkgwatch
  adds at these tiers is diff-and-alert (notice something changed, surface
  the new hash for a human to look at), not verify-and-trust.

The post-build version sanity check (see Architecture below) runs
regardless of tier — it's a correctness gate on the build itself, not part
of the trust-tier judgment, and doesn't change this tiering.

### Surfacing trust, not just gating on it

- Every tracked package carries an explicit tier + one-line justification
  (e.g. "minisign, key pinned 2024-03" vs. "same-domain sha256, no
  independent signer") in a metadata file alongside the generated PKGBUILD —
  something `repo-add`/pacman don't touch, but that a human or `pkgwatch
  audit` can read.
- `pkgwatch audit` (or similar) lists all tracked packages sorted
  worst-tier-first, so weak links don't hide among strong ones in a repo
  that otherwise looks uniformly trustworthy.

## Scaling to many packages

At a handful of tracked packages, several design choices above are
invisible non-issues. At dozens, they become real:

- **Review-queue fatigue.** Most real-world packages will land in tiers
  4–6 (bare GitHub release, no signing, is the common case, not the
  exception). If every release of every tracked package produces one
  review-and-approve event, the queue turns into something rubber-stamped
  to clear it — which degrades the already-weak tier 4–6 review into pure
  theater, worse than the single-package case. Mitigation: batch same-tier,
  low-signal changes (patch-version bump, no maintainer/key change) into a
  digest, and reserve individual review prompts for changes that look more
  significant (new signing key, jump of more than one minor version, new
  maintainer/publisher identity where that's knowable).
- **Per-source check cadence, not one global interval.** The original
  motivating problem — some software moves weekly, some quarterly — argues
  against checking everything on the same timer tick. See "Check method"
  below for the GitHub-specific answer; other source types likely want an
  explicit fast/normal/slow interval field per package rather than one
  daemon-wide interval.
- **Config as a directory, not one file.** `packages.d/*.toml` (one file
  per package), loaded as a directory — same convention as
  sudoers.d/systemd drop-ins — scales better than a single growing TOML
  file: easier to add/remove/diff one package, plays nicer with putting
  the config itself under version control.
- **`pkgwatch audit` becomes core, not peripheral.** At 3 packages you
  remember the trust tiers by heart. At 30 you don't. The audit/surfacing
  command (see "Surfacing trust," above) is what keeps the weak tiers safe
  to have around at all once the list is too big to hold in your head.
- **Local repo retention.** `makepkg`/`repo-add` output accumulates. Needs
  a "keep last N versions per package" prune step, or disk fills quietly
  over time.
- **Rate limits become real.** Enough GitHub-sourced packages checked on
  the same schedule can hit unauthenticated API limits — argues for an
  optional auth token in config, and/or staggering check times across
  packages rather than firing every check on the same tick.
- **Template reuse matters more.** Already an open schema question below,
  but at scale "a small fixed set of parameterized PKGBUILD shapes" clearly
  wins over "bespoke template per package" on maintenance-burden grounds
  alone, not just taste.

### Check method: polling vs. push (GitHub specifically)

True server-initiated push isn't available for repos you don't own —
GitHub webhooks require admin access on the repo being watched, which
rules them out for upstream projects you're only consuming. A third-party
relay (e.g. newreleases.io) could convert this into a webhook on your end,
but that requires a publicly reachable HTTPS receiver on this box, which
cuts against the existing WireGuard-only/no-public-SSH posture for real
inbound exposure and a purely cosmetic latency win — refreshing a local
pacman repo doesn't need sub-minute notification.

The practical middle ground, outbound-only:

- **`github-atom` check method**: poll `https://github.com/<owner>/<repo>/releases.atom`.
  Public, unauthenticated, and (worth reconfirming at implementation time)
  historically not counted against the REST API rate limit — cheap enough
  to poll every few minutes, getting close to push-latency for the
  GitHub-hosted slice of tracked packages without any inbound exposure.
- **`github-api` check method**: for anything the Atom feed doesn't cover
  (asset-level metadata, attestations), use conditional GETs
  (`If-None-Match`/ETag) against `api.github.com` — a `304 Not Modified`
  historically didn't consume rate-limit quota either, so frequent polling
  stays cheap even on the real API.
- Non-GitHub sources still need the per-package interval field above;
  there's no equivalent free-to-poll feed for most of them.

## Config schema (draft)

```toml
[package.uv]
source = "github-release"
repo = "astral-sh/uv"
asset_pattern = "uv-x86_64-unknown-linux-gnu.tar.gz"
check_method = "github-atom"   # or "github-api"; see Scaling > Check method
check_interval = "5m"          # per-package, not a global daemon interval

# Verified 2026-09-11 against the real repo: uv publishes GitHub
# build-provenance attestations (sigstore bundle) for every release asset
# — tier 2, not the tier-4 same-origin-sha256 originally guessed here.
# Checked via `gh attestation verify` rather than reimplementing sigstore
# verification in Rust. Tier is *derived* from `method`, not stored
# separately — the PoC found that storing both invites a tier/method
# mismatch that would mean nothing (see `Verification::tier()` in the PoC).
[package.uv.verification]
method = "github-attestation"

# Post-build sanity check — correctness only, not a security control.
# Runs the built binary and confirms it reports the version pkgwatch
# believes it just built; mismatch blocks publish.
[package.uv.sanity_check]
command = "uv --version"
version_regex = 'uv (\d+\.\d+\.\d+)'

# Tier 4 example — same-origin checksum only, proves transport integrity,
# not authorship (this is what the uv example above was, until checked):
[package.otherpkg]
source = "github-release"
repo = "someorg/otherpkg"
asset_pattern = "otherpkg-x86_64-unknown-linux-gnu.tar.gz"

[package.otherpkg.verification]
method = "same-origin-sha256"
checksum_asset_pattern = "otherpkg-x86_64-unknown-linux-gnu.tar.gz.sha256"

# Tier 1 example — not yet implemented in the PoC (only
# same-origin-sha256 and github-attestation exist so far):
[package.somepkg]
source = "url-with-version-regex"
url = "https://example.com/downloads/"
version_regex = 'somepkg-(\d+\.\d+\.\d+)\.tar\.gz'

[package.somepkg.verification]
method = "minisign"
pinned_key = "RWQ...base64pubkey..."
```

**PoC status** (see `src/`, `packages.d/uv.toml`, `packages.d/scaleway-cli.toml`):
implements `repo`, `asset_pattern`, and `verification.method`
(`same-origin-sha256` | `github-attestation` only), loaded from
`packages.d/*.toml`. Confirmed working end to end against two real repos:

- `astral-sh/uv` — `github-atom` feed → fetch → `gh attestation verify`
  (tier 2) → state persisted so a second run reports "up to date."
- `scaleway/scaleway-cli` — first live exercise of `same-origin-sha256`
  (tier 4). Verified upstream ships no build-provenance attestations
  (`attestations` API 404s), so this is genuinely tier 4, not an
  under-verified tier 2. Surfaced two schema/implementation gaps beyond
  what uv exercised, both now handled:
  - Release asset names embed the version
    (`scaleway-cli_2.62.0_linux_amd64`), unlike uv's static names.
    `asset_pattern`/`checksum_asset_pattern` now support a `{version}`
    placeholder, substituted via `checker::version_from_tag` (which also
    strips a tag's leading `v`, since scaleway-cli tags `vX.Y.Z` but
    filenames use the bare version).
  - Checksums ship as one combined `SHA256SUMS` (one line per platform
    asset) rather than a per-asset file like uv's — the verifier now
    matches the line by filename instead of assuming a single-hash file.
  - Separately, scaleway-cli's Atom feed lists a `vX.Y.Z-dbg1` tag newest,
    with no real Release object behind it (`releases/tags/<tag>` 404s) —
    `checker::latest_github_release` now confirms each feed candidate
    against the releases API in feed order rather than trusting the first
    entry outright.

Tier 4-6 packages (scaleway-cli included) are not auto-published — see
Build/publish/review-queue below. `source`, `check_method`,
`check_interval`, and `sanity_check` are still schema sketch, not yet read
by the code — the PoC only knows how to check GitHub-release sources.
Build/publish/review-queue (`makepkg`, `repo-add`, tier 4–6 human review)
are not implemented yet; a tier 4–6 pass currently just logs "flagging for
review" and stops.

Open questions on the schema:

- How much of `nvchecker`'s source-type taxonomy (github, gitlab, pypi,
  crates.io, regex, htmlparser, ...) to reimplement vs. shell out to
  `nvchecker` itself for the version-check step and own only the
  verification + publish pipeline.
- PKGBUILD generation: full Jinja-style templates per package vs. a small
  fixed set of PKGBUILD "shapes" (single binary tarball, cargo-install,
  etc.) parameterized by the config.
- Where the local pacman repo lives and how it's registered in
  `pacman.conf` (one-time manual setup step vs. something pkgwatch manages).
- Failure/alerting channel for tier 4–6 change events — log only, or a
  notification hook (this box already has a wofi/Mako notification setup —
  see `project_wofi_notification_picker` in Claude's memory).
- ~~Config layout: single TOML vs. `packages.d/*.toml` directory~~ —
  resolved: PoC loads `packages.d/*.toml` directly.
- Digest/batching rules for low-signal tier 4–6 changes (see Scaling,
  above) — what counts as "low-signal" needs a concrete definition, not
  just "not a major version bump."

## Architecture sketch

- **Config loader**: parses `packages.d/*.toml` into an in-memory package
  list. *(Implemented — `src/config.rs`.)*
- **Checker**: per source type, resolves "what's the latest version" —
  likely reuses `nvchecker`'s logic/sources conceptually for non-GitHub
  sources eventually. For GitHub sources, prefers the `github-atom` feed
  (see Scaling > Check method) over unconditional REST polling.
  *(Implemented for GitHub only — `src/checker.rs` regex-matches the first
  `releases/tag/<tag>` link in the feed rather than doing a full XML parse;
  fine while the feed's newest-entry-first shape holds, revisit if that
  ever changes. `check_interval`/per-package cadence not wired up yet —
  the PoC is a single one-shot run, not a scheduled loop.)*
- **Fetcher**: downloads the artifact (and any checksum/signature/
  attestation companion) for a resolved version. *(Implemented —
  `src/fetcher.rs`, via the GitHub releases API; exact asset-name match,
  not a glob.)*
- **Verifier**: tier-specific verification implementations, dispatched via
  a `Verification` enum matched on `method` (an internally-tagged serde
  enum) rather than a trait — simpler while there are only two methods;
  revisit as a trait if the method count grows. Returns a tier + pass/fail
  + justification string; tier is derived from `method`, never configured
  separately. *(Implemented for `same-origin-sha256` and
  `github-attestation` — `src/verifier.rs`. The latter shells out to `gh
  attestation verify` rather than reimplementing sigstore verification.)*
- **Builder**: for tiers 1–3 on pass, generates/updates the PKGBUILD
  (strict validation on any upstream-controlled string — version, filename
  — before it touches generated shell content; never unescaped
  interpolation) and runs `makepkg`.
- **Sanity checker**: after a successful build, runs the package's
  declared `sanity_check.command` against the built artifact and confirms
  the reported version matches what pkgwatch believes it just built.
  Mismatch = fail loud, do not publish. This is a correctness check, not a
  security control — it catches checker bugs and mangled/wrong-artifact
  downloads, not malicious releases.
- **Publisher**: runs `repo-add` against the local repo, only after the
  sanity check passes.
- **Reviewer queue**: for tiers 4–6, records the detected change instead of
  auto-building; a separate `pkgwatch review` command lets a human
  approve/reject, which then triggers the build → sanity-check → publish
  steps above.
- **Scheduling**: systemd `.service` (oneshot) + `.timer` running it
  periodically, matching the pattern already used for other periodic tasks
  on this box.

## Prior art / reference points

- `nvchecker` — version-check-only, no verification or publish step.
- `updpkgsums` (pacman-contrib/devtools) — checksum refresh only, manual
  trigger.
- `aurutils` — local repo + AUR build automation, but AUR itself carries no
  stronger verification guarantee than what each PKGBUILD maintainer does.
- `repology` — cross-distro version tracking, no verification/publish.
- GitHub artifact attestations (`gh attestation verify`) — tier 2 building
  block for GitHub-hosted releases.

## Status / next steps

- [x] Scope decided: personal middle-ground tool for a curated package
      list, not a general supply-chain-security framework (see Scope
      above). Downgrade attacks and compromised-vendor-pipeline defense
      are explicit non-goals; per-user scale to many packages is *not* a
      non-goal (see Scaling to many packages, above).
- [x] Check method decided for GitHub sources: `github-atom`/conditional
      `github-api` polling, outbound-only. Inbound webhooks explicitly
      rejected — no repo-admin access on upstreams, and a relay-based
      alternative would require a public receiver this box's networking
      posture deliberately avoids.
- [x] `packages.d/*.toml` config layout implemented (`src/config.rs`).
- [x] First PoC iteration, working end to end against the real
      `astral-sh/uv` repo: `github-atom` check → GitHub-API fetch →
      `github-attestation` (tier 2) verify via `gh attestation verify` →
      state persisted so re-runs report "up to date." Confirmed uv
      actually ships attestations, correcting the spec's original tier-4
      guess for it. Run: `cargo run` from the project root.
- [x] `same-origin-sha256` exercised against a real package:
      `scaleway/scaleway-cli`, tracked via `packages.d/scaleway-cli.toml`
      (added because Manjaro's `extra` scaleway-cli lags upstream). Tier 4
      confirmed correct — no build-provenance attestations upstream.
      Required adding `{version}`-placeholder support to `asset_pattern`/
      `checksum_asset_pattern`, filename-matched parsing of combined
      multi-asset checksum files, and having `latest_github_release`
      confirm each Atom-feed candidate against the releases API (this
      repo's newest feed entry, a `-dbg1` tag, has no real Release behind
      it). Still just flags for human review, same as any tier 4-6 pass —
      not auto-installed; see the unchecked build/publish item below.
- [ ] Not yet implemented: build (PKGBUILD generation + `makepkg`),
      publish (`repo-add`), reviewer queue for tier 4–6, scheduling/
      `check_interval`, non-GitHub sources, `minisign`/tier-1 method.
- [ ] Refine config schema further (see open questions above), including
      the `sanity_check` block per package.
- [ ] Decide version-check strategy for non-GitHub sources: shell out to
      `nvchecker` vs. own implementation.
- [ ] Implement PKGBUILD generation with strict upstream-string validation
      from day one (see Builder, above) — cheap to do right up front,
      expensive to retrofit.
- [ ] Next PoC iteration: carry the verified `uv` artifact through
      build → sanity-check → `repo-add` publish, closing the loop to an
      actual local pacman repo `pacman -Syu` can pick up.
- [ ] Decide on project home: local-only for now, or push to
      code.austinschaefer.com (Forgejo) once the spec settles.