Design for scale: many tracked packages, not multi-user

Corrects the earlier non-goal framing: per-user scale (tracking dozens of
packages) is explicitly in scope, distinct from multi-user/adversarial
config trust, which stays out. Adds a Scaling section covering review-queue
fatigue at volume, per-package check cadence, packages.d/ config layout,
audit-as-core, local repo retention, and GitHub rate limits/staggering.

Also settles the GitHub push-notification question: no true webhook push
for repos we don't own, and a relay-based alternative would need a public
inbound receiver this box's WireGuard-only posture deliberately avoids.
Settles on outbound-only github-atom/conditional github-api polling
instead, added to the config schema as check_method/check_interval.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01A2FEut5tVMNjeVjqhgVZbr

SPEC.md

@@ -48,8 +48,13 @@ than blindly piping an install script to `sh`.
   is curated by the one person running the daemon on their own machine;
   config/state file integrity relies on normal filesystem permissions, not
   a hardened trust boundary. Key-pinning friction on rotation (see tiers
-  below) is acceptable, even desirable, at this scale — it's a personal,
+  below) is acceptable, even desirable, regardless of list size — it's
-  low-volume list, not something that needs to scale painlessly.
+  about who's allowed to add a trust decision, not about volume.
+
+This is a narrower non-goal than "doesn't need to scale" — see **Scaling to
+many packages**, below. If pkgwatch actually solves the release-cadence
+problem, the natural outcome is tracking many packages, not a handful, and
+the design should hold up under that.
 
 ## Vision
 
@@ -78,10 +83,10 @@ that those tools don't provide.
 Per the Scope above, the goal is to raise the bar above curl|sh where the
 vendor gives us something to check — not to build airtight supply-chain
 defense. The central design problem within that goal: from the install UX,
-a cryptographically strong
+a cryptographically strong verification and a "trust-me-bro" same-domain
-verification and a "trust-me-bro" same-domain checksum look identical. The
+checksum look identical. The tool's job is to make that difference legible
-tool's job is to make that difference legible instead of laundering every
+instead of laundering every package into an undifferentiated "verified"
-package into an undifferentiated "verified" bucket.
+bucket.
 
 Tiers, strongest to weakest:
 
@@ -131,6 +136,74 @@ of the trust-tier judgment, and doesn't change this tiering.
   worst-tier-first, so weak links don't hide among strong ones in a repo
   that otherwise looks uniformly trustworthy.
 
+## Scaling to many packages
+
+At a handful of tracked packages, several design choices above are
+invisible non-issues. At dozens, they become real:
+
+- **Review-queue fatigue.** Most real-world packages will land in tiers
+  4–6 (bare GitHub release, no signing, is the common case, not the
+  exception). If every release of every tracked package produces one
+  review-and-approve event, the queue turns into something rubber-stamped
+  to clear it — which degrades the already-weak tier 4–6 review into pure
+  theater, worse than the single-package case. Mitigation: batch same-tier,
+  low-signal changes (patch-version bump, no maintainer/key change) into a
+  digest, and reserve individual review prompts for changes that look more
+  significant (new signing key, jump of more than one minor version, new
+  maintainer/publisher identity where that's knowable).
+- **Per-source check cadence, not one global interval.** The original
+  motivating problem — some software moves weekly, some quarterly — argues
+  against checking everything on the same timer tick. See "Check method"
+  below for the GitHub-specific answer; other source types likely want an
+  explicit fast/normal/slow interval field per package rather than one
+  daemon-wide interval.
+- **Config as a directory, not one file.** `packages.d/*.toml` (one file
+  per package), loaded as a directory — same convention as
+  sudoers.d/systemd drop-ins — scales better than a single growing TOML
+  file: easier to add/remove/diff one package, plays nicer with putting
+  the config itself under version control.
+- **`pkgwatch audit` becomes core, not peripheral.** At 3 packages you
+  remember the trust tiers by heart. At 30 you don't. The audit/surfacing
+  command (see "Surfacing trust," above) is what keeps the weak tiers safe
+  to have around at all once the list is too big to hold in your head.
+- **Local repo retention.** `makepkg`/`repo-add` output accumulates. Needs
+  a "keep last N versions per package" prune step, or disk fills quietly
+  over time.
+- **Rate limits become real.** Enough GitHub-sourced packages checked on
+  the same schedule can hit unauthenticated API limits — argues for an
+  optional auth token in config, and/or staggering check times across
+  packages rather than firing every check on the same tick.
+- **Template reuse matters more.** Already an open schema question below,
+  but at scale "a small fixed set of parameterized PKGBUILD shapes" clearly
+  wins over "bespoke template per package" on maintenance-burden grounds
+  alone, not just taste.
+
+### Check method: polling vs. push (GitHub specifically)
+
+True server-initiated push isn't available for repos you don't own —
+GitHub webhooks require admin access on the repo being watched, which
+rules them out for upstream projects you're only consuming. A third-party
+relay (e.g. newreleases.io) could convert this into a webhook on your end,
+but that requires a publicly reachable HTTPS receiver on this box, which
+cuts against the existing WireGuard-only/no-public-SSH posture for real
+inbound exposure and a purely cosmetic latency win — refreshing a local
+pacman repo doesn't need sub-minute notification.
+
+The practical middle ground, outbound-only:
+
+- **`github-atom` check method**: poll `https://github.com/<owner>/<repo>/releases.atom`.
+  Public, unauthenticated, and (worth reconfirming at implementation time)
+  historically not counted against the REST API rate limit — cheap enough
+  to poll every few minutes, getting close to push-latency for the
+  GitHub-hosted slice of tracked packages without any inbound exposure.
+- **`github-api` check method**: for anything the Atom feed doesn't cover
+  (asset-level metadata, attestations), use conditional GETs
+  (`If-None-Match`/ETag) against `api.github.com` — a `304 Not Modified`
+  historically didn't consume rate-limit quota either, so frequent polling
+  stays cheap even on the real API.
+- Non-GitHub sources still need the per-package interval field above;
+  there's no equivalent free-to-poll feed for most of them.
+
 ## Config schema (draft)
 
 ```toml
@@ -138,6 +211,8 @@ of the trust-tier judgment, and doesn't change this tiering.
 source = "github-release"
 repo = "astral-sh/uv"
 asset_pattern = "uv-x86_64-unknown-linux-gnu.tar.gz"
+check_method = "github-atom"   # or "github-api"; see Scaling > Check method
+check_interval = "5m"          # per-package, not a global daemon interval
 
 [package.uv.verification]
 tier = 4
@@ -177,6 +252,12 @@ Open questions on the schema:
 - Failure/alerting channel for tier 4–6 change events — log only, or a
   notification hook (this box already has a wofi/Mako notification setup —
   see `project_wofi_notification_picker` in Claude's memory).
+- Config layout: single TOML vs. `packages.d/*.toml` directory (see
+  Scaling, above) — probably directory-based from the start, since
+  retrofitting later means a migration step for no benefit.
+- Digest/batching rules for low-signal tier 4–6 changes (see Scaling,
+  above) — what counts as "low-signal" needs a concrete definition, not
+  just "not a major version bump."
 
 ## Architecture sketch
 
@@ -184,7 +265,10 @@ Open questions on the schema:
 - **Checker**: per source type, resolves "what's the latest version" —
   likely reuses `nvchecker`'s logic/sources conceptually, possibly shells
   out to it initially for the PoC rather than reimplementing every source
-  type in Rust.
+  type in Rust. For GitHub sources, prefers the `github-atom` feed or
+  conditional-GET `github-api` calls (see Scaling > Check method) over
+  plain unconditional REST polling. Respects each package's own
+  `check_interval` rather than a single daemon-wide tick.
 - **Fetcher**: downloads the artifact (and any checksum/signature/
   attestation companion) for a resolved version.
 - **Verifier**: tier-specific verification implementations behind a common
@@ -225,9 +309,15 @@ Open questions on the schema:
 - [x] Scope decided: personal middle-ground tool for a curated package
       list, not a general supply-chain-security framework (see Scope
       above). Downgrade attacks and compromised-vendor-pipeline defense
-      are explicit non-goals.
+      are explicit non-goals; per-user scale to many packages is *not* a
+      non-goal (see Scaling to many packages, above).
+- [x] Check method decided for GitHub sources: `github-atom`/conditional
+      `github-api` polling, outbound-only. Inbound webhooks explicitly
+      rejected — no repo-admin access on upstreams, and a relay-based
+      alternative would require a public receiver this box's networking
+      posture deliberately avoids.
 - [ ] Refine config schema further (see open questions above), including
-      the `sanity_check` block per package.
+      the `sanity_check` block per package and `packages.d/` layout.
 - [ ] Decide version-check strategy: shell out to `nvchecker` vs. own
       implementation, for the PoC.
 - [ ] Implement PKGBUILD generation with strict upstream-string validation