Commit graph

16 commits

Author SHA1 Message Date
d96b7a32f6 Merge pull request 'Close the loop: build, sanity-check, and publish' (#1) from worktree-build-publish-pipeline into master
All checks were successful
CI / build (push) Successful in 43s
CI / test (push) Successful in 3m38s
CI / audit (push) Successful in 13s
CI / coverage (push) Successful in 7m22s
Reviewed-on: #1
2026-09-18 10:56:35 +00:00
Austin Schaefer
0044eda532 Add a third builder shape for archives with no wrapping directory
All checks were successful
CI / build (pull_request) Successful in 1m13s
CI / test (pull_request) Successful in 3m46s
CI / audit (pull_request) Successful in 15s
CI / coverage (pull_request) Successful in 7m27s
Merging master's claude-code.toml onto this branch surfaced a real gap:
builder.rs only knew "bare binary download" and "tarball extracting into
a same-named directory" (uv). claude-code's tarball extracts a bare
`claude` file with no wrapping directory, and that inner filename
doesn't match the package name either — makepkg's package() failed with
"cannot stat .../claude-linux-x64/claude-code" (confirmed by actually
running the build).

Add Package::archive_binary_path, an explicit override for the
in-archive path builder.rs installs from, used verbatim when present
instead of the stem/binary_name convention. Set binary_name = "claude"
too, matching the box's actual command name (/opt/claude-code/bin/claude)
rather than the claude-code package name. Also added a sanity_check
block (claude --version), matching uv's pattern, confirmed against the
real built binary's output ("2.1.276 (Claude Code)").

Verified end to end against the real repo with PKGWATCH_REPO_DIR
pointed at a scratch dir: check -> fetch -> verify -> review --approve
-> build -> sanity-check -> publish all pass for claude-code v2.1.276.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
2026-09-18 12:46:28 +02:00
14e1cee36f Merge branch 'master' into worktree-build-publish-pipeline
All checks were successful
CI / build (pull_request) Successful in 1m6s
CI / test (pull_request) Successful in 3m58s
CI / audit (pull_request) Successful in 15s
CI / coverage (pull_request) Successful in 8m6s
2026-09-18 10:38:05 +00:00
Austin Schaefer
8694fa647f Track anthropics/claude-code releases
All checks were successful
CI / build (push) Successful in 59s
CI / test (push) Successful in 3m23s
CI / audit (push) Successful in 14s
CI / coverage (push) Successful in 6m31s
Same-origin-sha256 (tier 4) against SHASUMS256.txt — no GitHub
attestations upstream. A PGP signature over the checksum file also
ships (SHASUMS256.txt.sig) but pkgwatch has no minisign/GPG verifier
yet; revisit once tier 1 lands. Tracks the glibc x86_64 Linux build.

Confirmed working end to end: check -> fetch -> verify against the
real repo (v2.1.276 at time of writing).

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
2026-09-18 11:56:34 +02:00
Austin Schaefer
3e87282b21 Move high-level docs into docs/, matching doubleo7's convention
All checks were successful
CI / build (pull_request) Successful in 48s
CI / test (pull_request) Successful in 2m1s
CI / audit (pull_request) Successful in 9s
CI / coverage (pull_request) Successful in 3m53s
SPEC.md and ARCHITECTURE.md were sitting at the repo root alongside
Cargo.toml/Makefile.toml/packages.d — moved both into docs/ (doubleo7
already does this for its own supplementary docs, so this matches an
existing convention in the fleet rather than inventing a new one).

Updated every doc-comment cross-reference across src/*.rs and
Makefile.toml (23 references) to the new docs/SPEC.md / docs/
ARCHITECTURE.md paths. The two files' own cross-references to each other
didn't need changing — they're still same-directory relative references.

Also updated the project reference memory pointing at ARCHITECTURE.md's
location, so it doesn't go stale pointing at a path that no longer exists.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
2026-09-17 13:22:40 +02:00
Austin Schaefer
f55188e36f Fix issues from code review: shell-escaping gap, exit code, and more
All checks were successful
CI / build (pull_request) Successful in 51s
CI / test (pull_request) Successful in 3m35s
CI / audit (pull_request) Successful in 13s
CI / coverage (pull_request) Successful in 6m57s
A single-agent code review of this branch's diff (builder/pipeline/
publisher/sanity/hash + main/config/fetcher/state/verifier changes)
found six real issues, all fixed here:

- builder.rs: validate_shell_safe only rejected a literal single quote
  and newline, written for the single-quoted PKGBUILD fields. But
  asset_name (via install_source) and binary_name land in the install()
  line, which is necessarily double-quoted so ${srcdir}/${pkgdir} can
  expand — where $, backtick, and backslash are still live. Not
  currently exploitable (the one variable component, version, is already
  independently constrained by validate_pkgver's strict charset), but a
  latent gap relying on that coincidence rather than the validator
  actually covering its real use context. Widened the reject-list to
  cover both quoting styles, added regression tests including one at the
  generate_pkgbuild level. Corrected SPEC.md's "single-quoted" claim to
  match.
- pipeline.rs: a verification failure returned Ok(()) from
  process_package, so run_check never counted it as a failure and the
  process exited 0 even on a failed cryptographic/attestation check —
  exactly the event a monitoring setup (systemd OnFailure=, cron
  mail-on-error) most needs a non-zero exit to catch. Now bails, which
  run_check already treats as a package failure. Added an integration
  test against a mocked GitHub server exercising this exact path.
- builder.rs: find_built_package hardcoded the .pkg.tar.zst suffix, so a
  box with a different PKGEXT in makepkg.conf would report a false
  "makepkg failed" for a build that actually succeeded. Widened to match
  any .pkg.tar.* compression. Added direct unit tests (it had none).
- pipeline.rs: a newer tier 4-6 version silently overwrote a still-
  unreviewed older pending version with no indication anything was
  superseded. Now says so explicitly.
- hash.rs: builder/verifier each read a whole downloaded artifact into
  memory via std::fs::read just to hash it, doubling peak memory for no
  reason since the file's already on disk. Added sha256_hex_file,
  streamed in fixed-size chunks; both callers switched to it.
- Deduplicated two near-identical test-only "write an executable shell
  script" helpers (publisher.rs, sanity.rs) into a shared
  src/test_support.rs.

75 tests (was 63), cargo make ci clean. Re-verified end to end against
the real astral-sh/uv release after all six fixes — build, sanity check,
and publish into a scratch repo all still succeed.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
2026-09-17 12:06:40 +02:00
Austin Schaefer
3f30e0acc6 Add ARCHITECTURE.md and apply it to this PR's code
All checks were successful
CI / build (pull_request) Successful in 54s
CI / test (pull_request) Successful in 4m6s
CI / audit (pull_request) Successful in 17s
CI / coverage (pull_request) Successful in 6m19s
Researched current industry practice on code organization/maintainability
(Ousterhout's deep modules and information hiding, package-by-feature vs.
package-by-layer, functional-core/imperative-shell testability, tech-debt
prevention via ADR-equivalent inline rationale) and wrote it into
ARCHITECTURE.md as a set of concrete, project-specific rules rather than
a generic essay — each principle cites a real example already in this
codebase or fixed by this commit. Cross-linked from SPEC.md, which stays
about product design, not code organization.

Applied it to this PR's own code:

- Pulled process_package/fetch_and_verify/build_and_publish/run_review/
  approve out of main.rs into a new pipeline.rs. main.rs's own main() had
  grown to 278 lines and zero tests by treating "it's just the entry
  point" as an excuse to skip separating logic from wiring; now main.rs
  is argv dispatch only.
- Extracted decide_tier_action as a pure function (verification outcome +
  pending-state -> what to do), replacing dispatch logic that was
  previously inlined into a function that also made the real network/
  build calls. Four unit tests, no I/O, covering all four outcomes.
- Added a `//!` module doc comment to every file touched in this branch,
  each stating that module's one job in a sentence, per the "deep
  modules" principle the spec argues for.

Coverage's reported total drops (94% -> 78%) because pipeline.rs is
deliberately NOT excluded from it the way main.rs is, even though it's
mostly the same kind of untestable I/O orchestration — excluding it would
hide decide_tier_action's real unit-test coverage along with the untested
parts. Noted inline in Makefile.toml/ci.yml so the number doesn't look
like a quality regression at a glance.

Also added a project reference memory pointing at ARCHITECTURE.md rather
than duplicating its content there, per this session's own memory-hygiene
rules (architecture/conventions are derivable from the repo and shouldn't
be duplicated somewhere that can go stale).

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
2026-09-17 11:42:18 +02:00
Austin Schaefer
6305743e4d Check the custom repo is registered in pacman.conf before building
All checks were successful
CI / build (pull_request) Successful in 1m22s
CI / test (pull_request) Successful in 3m50s
CI / audit (pull_request) Successful in 14s
CI / coverage (pull_request) Successful in 9m2s
PR feedback: the local repo isn't guaranteed to exist on every box this
runs on, so it shouldn't just be assumed. publisher::ensure_registered
checks /etc/pacman.conf for an active [<repo_name>] section before a
build even starts, failing fast with the exact snippet to add if it's
missing — instead of spending several seconds on a makepkg build that
would succeed and then publish into a repo pacman never syncs from.

The repo directory and its database file were already self-healing
(publish creates the dir if missing, repo-add creates the db on first
run) — the actual gap was the pacman.conf registration, which can't be
made self-healing without root, so this fails loud with instructions
instead of trying to write to /etc/pacman.conf itself.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
2026-09-17 11:30:43 +02:00
Austin Schaefer
68fa648010 Close the loop: build, sanity-check, and publish for the first time
All checks were successful
CI / build (pull_request) Successful in 13m14s
CI / test (pull_request) Successful in 3m35s
CI / coverage (pull_request) Successful in 9m23s
CI / audit (pull_request) Successful in 13s
Implements the last unimplemented pipeline stage from SPEC.md: PKGBUILD
generation + makepkg (builder.rs), a post-build version sanity check
(sanity.rs), and repo-add publishing (publisher.rs), wired into main.rs
for both the tier 1-3 auto-publish path and a new tier 4-6 review queue
(`pkgwatch review` / `pkgwatch review <name> --approve`, persisted via
state::{load,save,clear}_pending_version, tracked separately from
last-published-version since approving one release isn't a standing
auto-publish grant for future ones).

Publishing targets an existing, already-registered local pacman repo
(~/.local/share/pacman/custom, `[custom]` in /etc/pacman.conf) rather
than one pkgwatch invents — found already in real use for a hand-packaged
AppImage, which resolves SPEC's open question on where the repo lives
without pkgwatch ever touching pacman.conf. Publishing stops at
`repo-add`; actually installing/upgrading (`pacman -Syu`/`pacman -S`) is
left to the operator, not run automatically.

Getting a real second package (scaleway-cli, tier 4) through the new
pipeline immediately surfaced a real gap: its pacman package is named
`scaleway-cli` but the actual binary is `scw` (confirmed via `pacman -Ql`
against the currently-installed extra package) — without a way to
declare that, the build would install alongside extra's package under
the wrong name instead of shadowing it. Added `Package::binary_name`
(config.rs) to cover it.

Every upstream-controlled string (version, asset name, download URL)
is validated before it touches generated shell content in the PKGBUILD
template — rejects anything containing a single quote or newline, since
values are embedded in single-quoted bash strings.

Verified for real, end to end: uv (tier 2) auto-built and published
against the real astral-sh/uv release with no human step; scaleway-cli
(tier 4) queued for review, then approved via `pkgwatch review
scaleway-cli --approve`, which re-verified, built, and published it —
confirmed the built package contains exactly usr/bin/scw. Both landed in
the real custom repo's database. Left scaleway-cli's real-repo review
pending rather than approving it myself: the tier 4-6 gate exists for a
human judgment call, not the agent's.

69 tests, cargo make ci clean (fmt, clippy, complexity, coverage, audit).

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
2026-09-17 11:13:13 +02:00
Austin Schaefer
13a1381bdf Add shift-left quality gates: cognitive complexity, coverage, dependency fix
All checks were successful
CI / build (push) Successful in 13m17s
CI / test (push) Successful in 3m59s
CI / coverage (push) Successful in 9m22s
CI / audit (push) Successful in 16s
Adds two new gates to the existing format/lint/test/audit pipeline
(Makefile.toml `cargo make ci`, .forgejo/workflows/ci.yml):

- Cognitive complexity via clippy's nursery cognitive_complexity lint
  (clippy.toml, threshold 15), scoped to --bins so test code's naturally
  higher branch count doesn't get gated. Went with this over the closest
  real cyclomatic-complexity tool (rust-code-analysis-cli) because that
  crate hasn't shipped a release since Jan 2023.
- Test coverage via cargo-llvm-cov, chosen over cargo-tarpaulin for
  friendlier behavior in containerized/dind CI (no ptrace). Report-only
  for now (no --fail-under-lines) since a real threshold needs real usage
  data first — see below. main.rs is excluded: it's orchestration glue
  exercised by the real end-to-end `cargo run`, not unit tests.

Getting both gates running required writing pkgwatch's first tests
(previously zero). To make the GitHub-facing modules unit-testable
without hitting real github.com/api.github.com, added `GithubEndpoints`
(src/github.rs) so checker/fetcher/verifier take injectable base URLs,
and added mockito + tempfile as dev-dependencies. Result: 27 tests,
94% region / 96% line coverage excluding main.rs.

Also: cargo audit (now wired into `cargo make ci`) immediately caught a
real, currently-open advisory (RUSTSEC-2026-0285, published days ago) in
the transitive rustls dependency — bumped 0.23.44 -> 0.23.45 to clear it.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
2026-09-17 09:16:14 +02:00
Austin Schaefer
7f4bb4cccf Track scaleway-cli, add version-templated assets and combined checksums
Wire up scaleway/scaleway-cli as a watched package (extra's copy lags
weeks behind upstream). Getting a real second package running exposed
gaps uv's config never hit:

- asset_pattern/checksum_asset_pattern now support a {version}
  placeholder for release assets that embed the version in the
  filename (checker::version_from_tag strips a tag's leading `v`).
- same-origin-sha256 now matches the checksum line by filename instead
  of assuming a single-hash file, to support combined multi-asset
  checksum files like scaleway-cli's SHA256SUMS.
- latest_github_release now confirms each Atom-feed candidate against
  the releases API instead of trusting the newest entry outright —
  scaleway-cli publishes a `-dbg1` tag with no real Release object that
  otherwise sorts newest in the feed.

Confirmed correct against the real repo: no attestations upstream, so
tier 4 (same-origin-sha256) applies, and the checker correctly skips
the dbg1 tag to land on the real latest release.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
2026-09-17 09:00:53 +02:00
Austin Schaefer
388096264b Set up project tooling to match the rest of ~/dev's Rust fleet
Surveyed sporah/doubleo7/feedsignal/uy-immigration-watcher/notif-picker
for conventions and replicated the current dominant pattern rather than
inventing a new one:

- Forgejo CI (.forgejo/workflows/ci.yml): build/test/audit jobs on the
  rust-ci runner label, cargo+sccache caching, cargo fmt --check, cargo
  clippy --all-targets -- -D warnings, cargo audit. Matches
  sporah/doubleo7/feedsignal/uy-immigration-watcher; notif-picker's
  docker-label/manual-toolchain-install variant looks like an earlier
  iteration superseded by this one.
- Makefile.toml with format/format-check/lint/test/audit/build tasks and
  a `ci` task chaining them — copied from notif-picker's clean version,
  the only project that had this pattern. `cargo make ci` now runs the
  same checks locally that CI runs.
- Explicit empty [workspace] in Cargo.toml (doubleo7's pattern) so a
  nested git-worktree checkout can't accidentally inherit an ancestor
  directory's workspace manifest.
- rustfmt: no rustfmt.toml, matching every sibling project — default
  style is the established convention here, not an oversight.

New for this fleet, since nothing else in ~/dev has it: a git-native
pre-commit hook (.githooks/pre-commit, activated via `cargo make
install-hooks` / `git config core.hooksPath .githooks`) that runs `cargo
fmt` and re-stages whatever it reformats. Chose git's native hooksPath
over the pre-commit(.com) framework or cargo-husky — no extra runtime
dependency, hook is tracked and shareable, and nothing else here needs
Python. Kept to formatting only; clippy/audit stay in CI, which already
covers them and can run heavier checks than a commit hook should.

Fixed one clippy finding (useless format! in checker.rs) and reformatted
the existing code to match the now-enforced default rustfmt style.
`cargo make ci` passes clean.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01A2FEut5tVMNjeVjqhgVZbr
2026-09-11 10:13:29 +02:00
Austin Schaefer
bf46cbf073 First working iteration: check -> fetch -> verify for a real package
Rust PoC (cargo, packages.d/*.toml config) that checks astral-sh/uv's
GitHub Atom feed for a new release, fetches the matching asset via the
GitHub API, and verifies it. Confirmed live against the real repo: uv
actually ships GitHub build-provenance attestations (sigstore bundle) on
every release, so it's a tier-2 package, not the tier-4 same-origin-sha256
guessed in the original spec draft. Verified via `gh attestation verify`
rather than reimplementing sigstore in Rust. State persists across runs so
a second run correctly reports "up to date."

Also folds the finding back into SPEC.md: updates the uv example to
tier 2, derives tier from verification method instead of storing both
(avoids a tier/method mismatch that would mean nothing), marks the
packages.d/ layout question resolved, and updates Architecture/Status to
say what's actually implemented vs. still sketch (build/publish/review
queue, same-origin-sha256 against a live repo, scheduling, non-GitHub
sources, minisign are all still open).

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01A2FEut5tVMNjeVjqhgVZbr
2026-09-11 10:04:48 +02:00
Austin Schaefer
6361e9eb5e Design for scale: many tracked packages, not multi-user
Corrects the earlier non-goal framing: per-user scale (tracking dozens of
packages) is explicitly in scope, distinct from multi-user/adversarial
config trust, which stays out. Adds a Scaling section covering review-queue
fatigue at volume, per-package check cadence, packages.d/ config layout,
audit-as-core, local repo retention, and GitHub rate limits/staggering.

Also settles the GitHub push-notification question: no true webhook push
for repos we don't own, and a relay-based alternative would need a public
inbound receiver this box's WireGuard-only posture deliberately avoids.
Settles on outbound-only github-atom/conditional github-api polling
instead, added to the config schema as check_method/check_interval.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01A2FEut5tVMNjeVjqhgVZbr
2026-09-11 09:24:13 +02:00
Austin Schaefer
8443ecea69 Narrow scope: personal middle ground, not a security framework
Adds an explicit Scope/Non-Goals section (curated personal package list,
not adversarial-config or compromised-vendor-pipeline defense) and a
post-build version sanity check as its own pipeline stage, distinct from
the trust tiers — a correctness gate (does the build report the version
we expected), not a security control. Trims the next-steps list to match.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01A2FEut5tVMNjeVjqhgVZbr
2026-09-11 09:10:06 +02:00
Austin Schaefer
0fa994f60a Add initial design spec for declarative package-update watcher
Captures the verification trust-tier model (pinned-key signatures down to
trust-me-bro install scripts) and the daemon architecture discussed: check
-> fetch -> tier-aware verify -> PKGBUILD gen -> local repo publish, with
weak-tier changes routed to human review instead of auto-publish.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01A2FEut5tVMNjeVjqhgVZbr
2026-09-11 09:01:30 +02:00