diff --git a/SPEC.md b/SPEC.md index 05ff57f..096901b 100644 --- a/SPEC.md +++ b/SPEC.md @@ -48,8 +48,13 @@ than blindly piping an install script to `sh`. is curated by the one person running the daemon on their own machine; config/state file integrity relies on normal filesystem permissions, not a hardened trust boundary. Key-pinning friction on rotation (see tiers - below) is acceptable, even desirable, at this scale — it's a personal, - low-volume list, not something that needs to scale painlessly. + below) is acceptable, even desirable, regardless of list size — it's + about who's allowed to add a trust decision, not about volume. + +This is a narrower non-goal than "doesn't need to scale" — see **Scaling to +many packages**, below. If pkgwatch actually solves the release-cadence +problem, the natural outcome is tracking many packages, not a handful, and +the design should hold up under that. ## Vision @@ -78,10 +83,10 @@ that those tools don't provide. Per the Scope above, the goal is to raise the bar above curl|sh where the vendor gives us something to check — not to build airtight supply-chain defense. The central design problem within that goal: from the install UX, -a cryptographically strong -verification and a "trust-me-bro" same-domain checksum look identical. The -tool's job is to make that difference legible instead of laundering every -package into an undifferentiated "verified" bucket. +a cryptographically strong verification and a "trust-me-bro" same-domain +checksum look identical. The tool's job is to make that difference legible +instead of laundering every package into an undifferentiated "verified" +bucket. Tiers, strongest to weakest: @@ -131,6 +136,74 @@ of the trust-tier judgment, and doesn't change this tiering. worst-tier-first, so weak links don't hide among strong ones in a repo that otherwise looks uniformly trustworthy. +## Scaling to many packages + +At a handful of tracked packages, several design choices above are +invisible non-issues. At dozens, they become real: + +- **Review-queue fatigue.** Most real-world packages will land in tiers + 4–6 (bare GitHub release, no signing, is the common case, not the + exception). If every release of every tracked package produces one + review-and-approve event, the queue turns into something rubber-stamped + to clear it — which degrades the already-weak tier 4–6 review into pure + theater, worse than the single-package case. Mitigation: batch same-tier, + low-signal changes (patch-version bump, no maintainer/key change) into a + digest, and reserve individual review prompts for changes that look more + significant (new signing key, jump of more than one minor version, new + maintainer/publisher identity where that's knowable). +- **Per-source check cadence, not one global interval.** The original + motivating problem — some software moves weekly, some quarterly — argues + against checking everything on the same timer tick. See "Check method" + below for the GitHub-specific answer; other source types likely want an + explicit fast/normal/slow interval field per package rather than one + daemon-wide interval. +- **Config as a directory, not one file.** `packages.d/*.toml` (one file + per package), loaded as a directory — same convention as + sudoers.d/systemd drop-ins — scales better than a single growing TOML + file: easier to add/remove/diff one package, plays nicer with putting + the config itself under version control. +- **`pkgwatch audit` becomes core, not peripheral.** At 3 packages you + remember the trust tiers by heart. At 30 you don't. The audit/surfacing + command (see "Surfacing trust," above) is what keeps the weak tiers safe + to have around at all once the list is too big to hold in your head. +- **Local repo retention.** `makepkg`/`repo-add` output accumulates. Needs + a "keep last N versions per package" prune step, or disk fills quietly + over time. +- **Rate limits become real.** Enough GitHub-sourced packages checked on + the same schedule can hit unauthenticated API limits — argues for an + optional auth token in config, and/or staggering check times across + packages rather than firing every check on the same tick. +- **Template reuse matters more.** Already an open schema question below, + but at scale "a small fixed set of parameterized PKGBUILD shapes" clearly + wins over "bespoke template per package" on maintenance-burden grounds + alone, not just taste. + +### Check method: polling vs. push (GitHub specifically) + +True server-initiated push isn't available for repos you don't own — +GitHub webhooks require admin access on the repo being watched, which +rules them out for upstream projects you're only consuming. A third-party +relay (e.g. newreleases.io) could convert this into a webhook on your end, +but that requires a publicly reachable HTTPS receiver on this box, which +cuts against the existing WireGuard-only/no-public-SSH posture for real +inbound exposure and a purely cosmetic latency win — refreshing a local +pacman repo doesn't need sub-minute notification. + +The practical middle ground, outbound-only: + +- **`github-atom` check method**: poll `https://github.com///releases.atom`. + Public, unauthenticated, and (worth reconfirming at implementation time) + historically not counted against the REST API rate limit — cheap enough + to poll every few minutes, getting close to push-latency for the + GitHub-hosted slice of tracked packages without any inbound exposure. +- **`github-api` check method**: for anything the Atom feed doesn't cover + (asset-level metadata, attestations), use conditional GETs + (`If-None-Match`/ETag) against `api.github.com` — a `304 Not Modified` + historically didn't consume rate-limit quota either, so frequent polling + stays cheap even on the real API. +- Non-GitHub sources still need the per-package interval field above; + there's no equivalent free-to-poll feed for most of them. + ## Config schema (draft) ```toml @@ -138,6 +211,8 @@ of the trust-tier judgment, and doesn't change this tiering. source = "github-release" repo = "astral-sh/uv" asset_pattern = "uv-x86_64-unknown-linux-gnu.tar.gz" +check_method = "github-atom" # or "github-api"; see Scaling > Check method +check_interval = "5m" # per-package, not a global daemon interval [package.uv.verification] tier = 4 @@ -177,6 +252,12 @@ Open questions on the schema: - Failure/alerting channel for tier 4–6 change events — log only, or a notification hook (this box already has a wofi/Mako notification setup — see `project_wofi_notification_picker` in Claude's memory). +- Config layout: single TOML vs. `packages.d/*.toml` directory (see + Scaling, above) — probably directory-based from the start, since + retrofitting later means a migration step for no benefit. +- Digest/batching rules for low-signal tier 4–6 changes (see Scaling, + above) — what counts as "low-signal" needs a concrete definition, not + just "not a major version bump." ## Architecture sketch @@ -184,7 +265,10 @@ Open questions on the schema: - **Checker**: per source type, resolves "what's the latest version" — likely reuses `nvchecker`'s logic/sources conceptually, possibly shells out to it initially for the PoC rather than reimplementing every source - type in Rust. + type in Rust. For GitHub sources, prefers the `github-atom` feed or + conditional-GET `github-api` calls (see Scaling > Check method) over + plain unconditional REST polling. Respects each package's own + `check_interval` rather than a single daemon-wide tick. - **Fetcher**: downloads the artifact (and any checksum/signature/ attestation companion) for a resolved version. - **Verifier**: tier-specific verification implementations behind a common @@ -225,9 +309,15 @@ Open questions on the schema: - [x] Scope decided: personal middle-ground tool for a curated package list, not a general supply-chain-security framework (see Scope above). Downgrade attacks and compromised-vendor-pipeline defense - are explicit non-goals. + are explicit non-goals; per-user scale to many packages is *not* a + non-goal (see Scaling to many packages, above). +- [x] Check method decided for GitHub sources: `github-atom`/conditional + `github-api` polling, outbound-only. Inbound webhooks explicitly + rejected — no repo-admin access on upstreams, and a relay-based + alternative would require a public receiver this box's networking + posture deliberately avoids. - [ ] Refine config schema further (see open questions above), including - the `sanity_check` block per package. + the `sanity_check` block per package and `packages.d/` layout. - [ ] Decide version-check strategy: shell out to `nvchecker` vs. own implementation, for the PoC. - [ ] Implement PKGBUILD generation with strict upstream-string validation