schaefera/pkgwatch
1
0
Fork 0
Code Issues Pull requests 1 Projects Releases Packages Wiki Activity Actions 1

Close the loop: build, sanity-check, and publish #1

Merged
schaefera merged 7 commits from worktree-build-publish-pipeline into master 2026-09-18 10:56:36 +00:00
Conversation 0 Commits 7 Files changed 18 +1827 -140
18 changed files with 1827 additions and 140 deletions
Show stats Download patch file Download diff file Expand all files Collapse all files

7
.forgejo/workflows/ci.yml
View file

@ -110,9 +110,10 @@ jobs:
command -v cargo-llvm-cov >/dev/null 2>&1 || cargo install cargo-llvm-cov --locked command -v cargo-llvm-cov >/dev/null 2>&1 || cargo install cargo-llvm-cov --locked
# Reports coverage only — no --fail-under-lines yet. main.rs is # Reports coverage only — no --fail-under-lines yet. main.rs is
# excluded: thin orchestration glue exercised by the real end-to-end # excluded: thin argv dispatch exercised by the real end-to-end
# `cargo run` against live GitHub, not unit tests, so it's not a # `cargo run`, not unit tests, so it's not a meaningful signal here.
# meaningful signal here. See Makefile.toml > coverage-report. # See Makefile.toml > coverage-report for why pipeline.rs, despite
# being mostly untestable I/O orchestration too, stays included.
- name: Coverage - name: Coverage
run: cargo llvm-cov --ignore-filename-regex 'main\.rs' --summary-only run: cargo llvm-cov --ignore-filename-regex 'main\.rs' --summary-only

10
Makefile.toml
View file

@ -30,9 +30,15 @@ args = ["test"]
# same-named custom one. # same-named custom one.
# #
# Reports coverage only; not gated on a threshold yet — main.rs is thin # Reports coverage only; not gated on a threshold yet — main.rs is thin
# orchestration glue exercised by the real end-to-end `cargo run`, not unit # argv dispatch exercised by the real end-to-end `cargo run`, not unit
# tests, so it's excluded here rather than dragging the number down for # tests, so it's excluded here rather than dragging the number down for
# reasons unrelated to test quality. # reasons unrelated to test quality. pipeline.rs is deliberately NOT
# excluded even though it's mostly network/subprocess/filesystem
# orchestration too (hence its own low number) — its one pure decision
# function (decide_tier_action) is unit tested and should stay visible in
# this report; excluding the whole file would hide that signal along with
# the untested parts. See docs/ARCHITECTURE.md > "separate pure decision logic
# from I/O."
[tasks.coverage-report] [tasks.coverage-report]
command = "cargo" command = "cargo"
args = ["llvm-cov", "--ignore-filename-regex", "main\\.rs", "--summary-only"] args = ["llvm-cov", "--ignore-filename-regex", "main\\.rs", "--summary-only"]

131
docs/ARCHITECTURE.md Normal file
View file

@ -0,0 +1,131 @@
# pkgwatch — code organization
Status: written 2026-09-17, once the build/publish pipeline PR gave this
project enough real code to have actual conventions worth writing down,
instead of guessing at them in advance.
This is distinct from `SPEC.md`, which is the product design (what
pkgwatch does and why). This file is about how the *code* implementing
that design is organized, so it stays readable as it grows past PoC size
instead of quietly accumulating debt. Researched against current industry
practice rather than asserted from habit — see Further reading.
## Principles
1. **One module, one job — and say what it is, up front.**
Ousterhout's "deep modules": the best modules expose a lot of
functionality through a simple interface, hiding the complexity behind
it. The two failure modes he names — *change amplification* (one
conceptual change forces edits in many places) and *obscurity* (a
reader can't tell where responsibility lives) — are both symptoms of
modules that don't have one clear job.
**Rule**: every `src/*.rs` file opens with a `//!` doc comment stating
its one responsibility in a sentence. If it can't be one sentence, the
module is doing too much.
**Example already here**: `builder.rs`'s job is "turn an
already-downloaded, already-verified artifact into a built package." It
hides PKGBUILD templating, upstream-string validation, and the
`makepkg` invocation behind one `build()` call — none of that leaks to
callers.
2. **Organize by pipeline stage (feature), not by technical layer.**
The package-by-feature vs. package-by-layer research is consistent:
feature-based grouping gives high cohesion within a module and low
coupling between modules; layer-based grouping (`models/`, `utils/`,
`helpers/`) tends toward the opposite, and a single feature change ends
up touching files scattered across every layer.
**Rule**: modules are named after what they do in the pipeline
(`checker`, `fetcher`, `verifier`, `builder`, `sanity`, `publisher`,
`state`), not generic buckets. A new pipeline stage gets a new module
named after the stage, not a method bolted onto an existing one.
**Anti-example to keep watching for**: a `utils.rs` grab-bag. `hash.rs`
could look like one but isn't — it exists for exactly one piece of
shared logic (`sha256_hex`) that two real stages (`verifier`,
`builder`) both need, not as a place to dump unrelated helpers.
3. **Separate pure decision logic from I/O ("functional core, imperative
shell").**
A function that decides *and* does in the same body can't be tested
without standing up everything the "does" half touches — often a
network call, a subprocess, or the filesystem. Pulling the decision out
into its own pure function makes it trivially unit-testable and makes
the I/O half thin enough that it obviously matches the decision.
**Applied this PR**: `pipeline::process_package`'s tier dispatch
(publish now / still pending / newly pending / verification failed) was
originally inline in a function that also made the real network and
build calls. Pulled out into `decide_tier_action`, a pure function with
its own unit tests covering all four outcomes, no I/O involved.
4. **Every network, subprocess, filesystem-root, or environment boundary
is injectable.**
Same testability goal as #3, applied to the specific ways this program
reaches outside itself. A consistent shape beats ad hoc mocking invented
per call site.
**Already in force**: `GithubEndpoints` (checker/fetcher/verifier),
`repo_add_bin` and the pacman.conf path (publisher), `PKGWATCH_REPO_DIR`
(main, for manual dry runs against a scratch repo instead of the real
one). A new external call follows the same shape: production code calls
a thin wrapper with the real default; tests call the parameterized
version with a fake.
5. **`main.rs` is a dispatcher, not the program.**
Found by looking at this project's own `main.rs`: it grew to 278 lines
and zero tests over the course of one PR, because "it's just the entry
point" is an easy excuse to skip separating logic from wiring — even
though Rust doesn't actually stop you from unit-testing a binary
crate's `main.rs`. The Rust community convention of splitting
entry-point parsing from application logic exists precisely so the
logic ends up somewhere it's normal to test.
**Rule**: `main.rs` may parse `argv`, build shared clients, and print
output. It must not contain a pipeline decision, a network/subprocess
call, or anything with a test worth writing — that belongs in
`pipeline.rs`.
**Applied this PR**: moved `process_package`, `fetch_and_verify`,
`build_and_publish`, `run_review`, and `approve` out of `main.rs` into a
new `pipeline.rs`, leaving `main.rs` as argument dispatch only.
6. **Validate at the boundary, once — don't scatter checks.**
Already stated project-wide (see the user's global instructions: don't
validate scenarios that can't happen, validate at system boundaries).
**Example already here**: `builder.rs`'s `validate_pkgname`/
`validate_pkgver`/`validate_shell_safe` run once, at PKGBUILD-generation
time, against every upstream-controlled string — not sprinkled through
whatever code happens to produce those strings.
7. **Don't build generality the currently-tracked packages don't need.**
Already the load-bearing design principle in `SPEC.md` ("a small fixed
set of PKGBUILD shapes," "extend when a third real shape shows up").
Restated here because it's also a tech-debt principle in its own right:
speculative abstraction is debt too — every future reader has to
understand it whether or not it's ever exercised.
8. **Every non-obvious structural decision gets one sentence of "why,"
inline.**
Standard tech-debt-prevention advice is to keep Architecture Decision
Records; a single-crate personal tool doesn't need a `docs/adr/`
directory, but the same information — why this way and not the obvious
alternative — needs to live somewhere a future reader will actually see
it: the doc comment on the thing itself.
**Example already here**: `checker.rs`'s doc comment on
`latest_github_release` explains why the newest Atom-feed entry isn't
trusted outright (scaleway-cli's `-dbg1` tag has no real Release behind
it) — the reasoning lives right next to the code it justifies, not in a
commit message or a separate design doc no one will find later.
## What's machine-enforced vs. what isn't
`cargo make ci` (format, clippy, cognitive-complexity threshold, coverage,
audit) mechanically enforces what's checkable: style, a handful of lint
categories, a complexity ceiling, and that coverage doesn't quietly
regress. It does **not** enforce module cohesion, naming, or "is this
logic in the right module" — those stay code-review questions. Worth
being honest about that boundary rather than implying CI catches
everything above.
## Further reading
- [A Philosophy of Software Design — deep modules & information hiding, summary](https://medium.com/swlh/a-philosophy-of-software-design-by-john-ousterhout-4a00d0ff9f1c)
- [Package by feature vs. package by layer](https://medium.com/@felixnjunge78/package-by-feature-vs-package-by-layer-which-one-wins-11ee03921fed)
- [Coupling and cohesion as the foundations of a maintainable codebase](https://medium.com/@iamprovidence/coupling-and-cohesion-foundations-that-affect-your-entire-codebase-77d06d44af0d)
- [Rust module and crate organization best practices](https://softwarepatternslexicon.com/rust/idiomatic-rust-patterns/module-and-crate-organization-best-practices/)
- [Reducing technical debt in 2026 — IBM](https://www.ibm.com/think/insights/reduce-technical-debt)

132
SPEC.md → docs/SPEC.md
View file

@ -2,6 +2,10 @@
Status: design draft, pre-PoC. Captures the design discussion as of 2026-09-11. Status: design draft, pre-PoC. Captures the design discussion as of 2026-09-11.
This is the *product* design — what pkgwatch does and why. For how the
code implementing it is organized (module boundaries, testability
conventions, what CI does and doesn't enforce), see `ARCHITECTURE.md`.
## Problem ## Problem
Software not packaged by the distro (Arch/Manjaro here) usually gets installed Software not packaged by the distro (Arch/Manjaro here) usually gets installed
@ -242,6 +246,12 @@ asset_pattern = "otherpkg-x86_64-unknown-linux-gnu.tar.gz"
method = "same-origin-sha256" method = "same-origin-sha256"
checksum_asset_pattern = "otherpkg-x86_64-unknown-linux-gnu.tar.gz.sha256" checksum_asset_pattern = "otherpkg-x86_64-unknown-linux-gnu.tar.gz.sha256"
# binary_name: only needed when the installed binary's name differs from
# the pacman package name — e.g. real-world case, scaleway-cli's package
# is named scaleway-cli but its actual binary is `scw` (see
# packages.d/scaleway-cli.toml). Defaults to the package name.
binary_name = "otherbin"
# Tier 1 example — not yet implemented in the PoC (only # Tier 1 example — not yet implemented in the PoC (only
# same-origin-sha256 and github-attestation exist so far): # same-origin-sha256 and github-attestation exist so far):
[package.somepkg] [package.somepkg]
@ -281,13 +291,20 @@ implements `repo`, `asset_pattern`, and `verification.method`
against the releases API in feed order rather than trusting the first against the releases API in feed order rather than trusting the first
entry outright. entry outright.
Tier 4-6 packages (scaleway-cli included) are not auto-published — see `sanity_check` and `binary_name` are now real, implemented fields (see
Build/publish/review-queue below. `source`, `check_method`, Builder/Sanity checker above) — added `packages.d/uv.toml`'s and
`check_interval`, and `sanity_check` are still schema sketch, not yet read `packages.d/scaleway-cli.toml`'s own `sanity_check` blocks, and
by the code — the PoC only knows how to check GitHub-release sources. scaleway-cli's `binary_name = "scw"`. `source`, `check_method`, and
`check_interval` are still schema sketch, not yet read by the code — the
PoC only knows how to check GitHub-release sources, on a single one-shot
run rather than a scheduled loop.
Build/publish/review-queue (`makepkg`, `repo-add`, tier 4–6 human review) Build/publish/review-queue (`makepkg`, `repo-add`, tier 4–6 human review)
are not implemented yet; a tier 4–6 pass currently just logs "flagging for are now implemented too — see Builder/Sanity checker/Publisher/Reviewer
review" and stops. queue above and the Status entry below for the first full end-to-end run.
Tier 4-6 packages still don't auto-publish (by design, see Verification
trust tiers > Automation posture per tier); they queue for
`pkgwatch review <name> --approve`.
Open questions on the schema: Open questions on the schema:
@ -334,25 +351,65 @@ Open questions on the schema:
separately. *(Implemented for `same-origin-sha256` and separately. *(Implemented for `same-origin-sha256` and
`github-attestation` — `src/verifier.rs`. The latter shells out to `gh `github-attestation` — `src/verifier.rs`. The latter shells out to `gh
attestation verify` rather than reimplementing sigstore verification.)* attestation verify` rather than reimplementing sigstore verification.)*
- **Builder**: for tiers 1–3 on pass, generates/updates the PKGBUILD - **Builder**: for tiers 1–3 on pass, generates a PKGBUILD (strict
(strict validation on any upstream-controlled string — version, filename validation on every upstream-controlled string — version, asset name,
— before it touches generated shell content; never unescaped download URL — before it touches generated shell content; most fields
interpolation) and runs `makepkg`. are single-quoted, but the `install()` line necessarily uses double
quotes so `${srcdir}`/`${pkgdir}` expand, so the validation rejects `'`,
newline, `$`, backtick, *and* backslash — safe for either quoting style
rather than assuming a value only ever lands in one of them — never
unescaped interpolation) and runs `makepkg`. *(Implemented —
`src/builder.rs`. One fixed "prebuilt binary" PKGBUILD shape covers both
tracked packages so far: a bare-binary download (scaleway-cli) and a
tarball extracting to a same-named directory (uv) — see Scaling >
Template reuse. `Package.binary_name` (config.rs) covers the case where
the installed binary's name differs from the pacman package name, which
turned out to matter immediately: scaleway-cli's real binary is `scw`,
not `scaleway-cli` — confirmed by inspecting the currently-installed
`extra` package with `pacman -Ql`, not guessable from the repo name.
Without it the build would install alongside `extra`'s package instead
of shadowing it.)*
- **Sanity checker**: after a successful build, runs the package's - **Sanity checker**: after a successful build, runs the package's
declared `sanity_check.command` against the built artifact and confirms declared `sanity_check.command` — with the freshly built package's
the reported version matches what pkgwatch believes it just built. `usr/bin` prepended to `PATH`, so it exercises what was just built
Mismatch = fail loud, do not publish. This is a correctness check, not a rather than whatever's already installed system-wide — and confirms the
security control — it catches checker bugs and mangled/wrong-artifact reported version matches what pkgwatch believes it just built. Mismatch
downloads, not malicious releases. = fail loud, do not publish. This is a correctness check, not a security
- **Publisher**: runs `repo-add` against the local repo, only after the control — it catches checker bugs and mangled/wrong-artifact downloads,
sanity check passes. not malicious releases. *(Implemented — `src/sanity.rs`.)*
- **Publisher**: copies the built package into the local repo directory
and runs `repo-add`, only after the sanity check passes. *(Implemented —
`src/publisher.rs`. Targets an existing, already-registered local pacman
repo rather than one pkgwatch creates — this box already has one at
`~/.local/share/pacman/custom`, registered as `[custom]` in
`/etc/pacman.conf` (`SigLevel = Optional TrustAll`) and already in use
for a hand-packaged AppImage. But that repo directory/registration isn't
guaranteed to exist on every box this ever runs on, so it isn't just
assumed: `publisher::ensure_registered` checks `/etc/pacman.conf` for an
active `[<repo_name>]` section before a build even starts, failing fast
with the exact snippet to add if it's missing, rather than wasting a
`makepkg` build on a repo pacman will never sync from. The repo
*directory* and its database file, by contrast, are fully self-healing —
`publish` creates the directory if missing and `repo-add` creates the
database on its first run. What's deliberately not automatic, and can't
safely be: writing the `[section]` into `/etc/pacman.conf` itself — that
needs root, which this process doesn't have and shouldn't grab for
itself. Similarly, publish deliberately stops at `repo-add`: getting the
new version onto the running system is a separate, deliberate
`pacman -Syu`/`pacman -S <pkg>` step left to the operator, not run
automatically.)*
- **Reviewer queue**: for tiers 4–6, records the detected change instead of - **Reviewer queue**: for tiers 4–6, records the detected change instead of
auto-building; a separate `pkgwatch review` command lets a human auto-building; a separate `pkgwatch review` command lets a human
approve/reject, which then triggers the build → sanity-check → publish approve/reject, which then triggers the build → sanity-check → publish
steps above. steps above. *(Implemented — `state::{load,save,clear}_pending_version`
plus the `review`/`review <name> --approve` subcommands in `src/main.rs`.
Tracked separately from the last-published-version state: approving one
release doesn't mean future ones auto-publish. `--approve` re-verifies
before building rather than trusting a possibly-stale flag from an
earlier run. No reject/dismiss command yet — see Status below.)*
- **Scheduling**: systemd `.service` (oneshot) + `.timer` running it - **Scheduling**: systemd `.service` (oneshot) + `.timer` running it
periodically, matching the pattern already used for other periodic tasks periodically, matching the pattern already used for other periodic tasks
on this box. on this box. *(Not implemented — still a single one-shot `cargo run`.)*
## Prior art / reference points ## Prior art / reference points
@ -395,18 +452,33 @@ Open questions on the schema:
repo's newest feed entry, a `-dbg1` tag, has no real Release behind repo's newest feed entry, a `-dbg1` tag, has no real Release behind
it). Still just flags for human review, same as any tier 4-6 pass — it). Still just flags for human review, same as any tier 4-6 pass —
not auto-installed; see the unchecked build/publish item below. not auto-installed; see the unchecked build/publish item below.
- [ ] Not yet implemented: build (PKGBUILD generation + `makepkg`), - [x] Full pipeline closed end to end for the first time: check → fetch →
publish (`repo-add`), reviewer queue for tier 4–6, scheduling/ verify → build → sanity-check → publish, against two real packages.
`check_interval`, non-GitHub sources, `minisign`/tier-1 method. `uv` (tier 2) auto-built and published on the first run with no
- [ ] Refine config schema further (see open questions above), including human step. `scaleway-cli` (tier 4) queued for review, then
the `sanity_check` block per package. `pkgwatch review scaleway-cli --approve` re-verified, built, and
published it — confirmed the built package installs as
`/usr/bin/scw`, actually shadowing `extra`'s package rather than
installing alongside it under the wrong name. Both landed in the
real `~/.local/share/pacman/custom` repo's database
(`custom.db.tar.gz`), ready for `sudo pacman -Syu`/`sudo pacman -S`
— not run automatically. See Builder/Sanity checker/Publisher/
Reviewer queue above for what each piece does.
One cosmetic wrinkle, not a correctness issue: `makepkg` printed
`libfakeroot internal error: payload not recognized!` while
packaging scaleway-cli's large Go binary, but still produced a
correct package (verified: exactly `usr/bin/scw` plus standard
metadata) — looks like an environment quirk in this sandbox's
fakeroot, not something pkgwatch caused; revisit if a real build
ever actually fails on it.
- [ ] Not yet implemented: `pkgwatch review <name> --reject` (a pending
review can only be approved or left pending, not dismissed),
scheduling/`check_interval`, non-GitHub sources, `minisign`/tier-1
method, retention/pruning of old versions in the local repo (see
Scaling > Local repo retention), staggering/auth for GitHub API
rate limits at higher package counts.
- [ ] Refine config schema further (see open questions above).
- [ ] Decide version-check strategy for non-GitHub sources: shell out to - [ ] Decide version-check strategy for non-GitHub sources: shell out to
`nvchecker` vs. own implementation. `nvchecker` vs. own implementation.
- [ ] Implement PKGBUILD generation with strict upstream-string validation
from day one (see Builder, above) — cheap to do right up front,
expensive to retrofit.
- [ ] Next PoC iteration: carry the verified `uv` artifact through
build → sanity-check → `repo-add` publish, closing the loop to an
actual local pacman repo `pacman -Syu` can pick up.
- [ ] Decide on project home: local-only for now, or push to - [ ] Decide on project home: local-only for now, or push to
code.austinschaefer.com (Forgejo) once the spec settles. code.austinschaefer.com (Forgejo) once the spec settles.

14
packages.d/claude-code.toml
View file

@ -16,11 +16,25 @@
# no `{version}` placeholder is needed, same as uv's config. Tracking the # no `{version}` placeholder is needed, same as uv's config. Tracking the
# glibc x86_64 Linux build (`claude-linux-x64.tar.gz`), not the musl # glibc x86_64 Linux build (`claude-linux-x64.tar.gz`), not the musl
# variant, to match this machine. # variant, to match this machine.
#
# Archive shape doesn't match uv's or scaleway-cli's: the tarball extracts a
# bare `claude` file with no wrapping directory (confirmed via `tar tzvf`
# against the real v2.1.276 asset) — hence archive_binary_path below (see
# builder.rs's third shape). binary_name is also set explicitly to `claude`
# (the real upstream command name, not the `claude-code` package name) so
# the installed binary matches what this box already invokes as `claude`
# (see /opt/claude-code/bin/claude).
[package.claude-code] [package.claude-code]
repo = "anthropics/claude-code" repo = "anthropics/claude-code"
asset_pattern = "claude-linux-x64.tar.gz" asset_pattern = "claude-linux-x64.tar.gz"
binary_name = "claude"
archive_binary_path = "claude"
[package.claude-code.verification] [package.claude-code.verification]
method = "same-origin-sha256" method = "same-origin-sha256"
checksum_asset_pattern = "SHASUMS256.txt" checksum_asset_pattern = "SHASUMS256.txt"
[package.claude-code.sanity_check]
command = "claude --version"
version_regex = '(\d+\.\d+\.\d+) \(Claude Code\)'

11
packages.d/scaleway-cli.toml
View file

@ -7,11 +7,22 @@
# Releases ship one combined `SHA256SUMS` file (one line per platform # Releases ship one combined `SHA256SUMS` file (one line per platform
# asset) rather than a per-asset checksum file like uv's — verifier # asset) rather than a per-asset checksum file like uv's — verifier
# matches the line by filename. # matches the line by filename.
#
# binary_name = "scw": confirmed by checking the currently-installed
# `extra` package (`pacman -Ql scaleway-cli`) — the pacman package is
# named scaleway-cli but the actual binary it installs is `scw`. Without
# this, pkgwatch's build would install as /usr/bin/scaleway-cli, which
# would NOT shadow extra's /usr/bin/scw at all.
[package.scaleway-cli] [package.scaleway-cli]
repo = "scaleway/scaleway-cli" repo = "scaleway/scaleway-cli"
asset_pattern = "scaleway-cli_{version}_linux_amd64" asset_pattern = "scaleway-cli_{version}_linux_amd64"
binary_name = "scw"
[package.scaleway-cli.verification] [package.scaleway-cli.verification]
method = "same-origin-sha256" method = "same-origin-sha256"
checksum_asset_pattern = "SHA256SUMS" checksum_asset_pattern = "SHA256SUMS"
[package.scaleway-cli.sanity_check]
command = "scw version"
version_regex = 'Version\s+(\d+\.\d+\.\d+)'

4
packages.d/uv.toml
View file

@ -10,3 +10,7 @@ asset_pattern = "uv-x86_64-unknown-linux-gnu.tar.gz"
[package.uv.verification] [package.uv.verification]
method = "github-attestation" method = "github-attestation"
[package.uv.sanity_check]
command = "uv --version"
version_regex = 'uv (\d+\.\d+\.\d+)'

516
src/builder.rs Normal file
View file

@ -0,0 +1,516 @@
//! Turns an already-downloaded, already-verified artifact into a built
//! pacman package: generates a PKGBUILD, then runs `makepkg`. Hides all
//! PKGBUILD templating and upstream-string validation behind `build()`.
use crate::config::Package;
use crate::hash;
use anyhow::{Context, Result, bail};
use std::path::{Path, PathBuf};
use std::process::Command;
/// Archive extensions `makepkg` auto-extracts before `package()` runs.
/// Longest-first so `.tar.gz` isn't shadowed by a hypothetical `.gz` entry.
const ARCHIVE_EXTENSIONS: &[&str] = &[".tar.gz", ".tar.xz", ".tar.zst", ".tar.bz2", ".tgz", ".zip"];
/// Everything needed to generate and build a PKGBUILD for one release.
pub struct BuildRequest<'a> {
pub pkg_name: &'a str,
pub pkg: &'a Package,
pub version: &'a str,
pub repo: &'a str,
pub asset_name: &'a str,
pub download_url: &'a str,
pub artifact_path: &'a Path,
}
pub struct BuildResult {
/// The built `.pkg.tar.zst`, ready for `publisher::publish`.
pub package_path: PathBuf,
/// `makepkg`'s package staging directory (`$pkgdir`), still present
/// after a successful build — lets `sanity` exercise the freshly built
/// binary without installing it system-wide first.
pub pkgdir: PathBuf,
}
/// Generates a PKGBUILD around an already-downloaded, already-verified
/// artifact, then runs `makepkg` in `build_dir`.
///
/// Deliberately one fixed "prebuilt binary" shape, not a templating engine
/// — see docs/SPEC.md > Scaling > Template reuse. Covers the shapes
/// currently-tracked packages actually need: a bare-binary download
/// (scaleway-cli), a tarball containing a same-named directory (uv), and a
/// tarball with no wrapping directory at all whose inner filename doesn't
/// match the package name (claude-code — see `Package::archive_binary_path`).
/// Extend when a fourth real shape shows up rather than guessing at
/// generality now.
pub fn build(req: &BuildRequest, build_dir: &Path) -> Result<BuildResult> {
let pkgbuild = generate_pkgbuild(req)?;
std::fs::create_dir_all(build_dir)
.with_context(|| format!("creating build dir {}", build_dir.display()))?;
std::fs::write(build_dir.join("PKGBUILD"), pkgbuild)?;
// makepkg looks for the source file by its declared name next to
// PKGBUILD; pre-seed it with the copy pkgwatch already downloaded and
// verified so makepkg's own sha256 check passes without re-fetching
// from the network (and without trusting the network a second time).
std::fs::copy(req.artifact_path, build_dir.join(req.asset_name))?;
let status = Command::new("makepkg")
.args(["--noconfirm", "--force"])
.current_dir(build_dir)
.status()
.context("running makepkg (is base-devel installed?)")?;
if !status.success() {
bail!("makepkg failed for {} {}", req.pkg_name, req.version);
}
let package_path = find_built_package(build_dir, req.pkg_name, req.version)?;
let pkgdir = build_dir.join("pkg").join(req.pkg_name);
Ok(BuildResult {
package_path,
pkgdir,
})
}
/// Builds the PKGBUILD text for `req`, validating every upstream-controlled
/// string first (see docs/SPEC.md > Architecture > Builder: "strict validation
/// on any upstream-controlled string ... never unescaped interpolation").
/// Pure and side-effect-free so it's testable without invoking `makepkg`.
fn generate_pkgbuild(req: &BuildRequest) -> Result<String> {
validate_pkgname(req.pkg_name)?;
validate_pkgver(req.version)?;
validate_shell_safe("asset name", req.asset_name)?;
validate_shell_safe("download url", req.download_url)?;
validate_shell_safe("repo", req.repo)?;
let binary_name = req.pkg.binary_name(req.pkg_name);
validate_pkgname(binary_name)?;
let sha256 = hash::sha256_hex_file(req.artifact_path)?;
let install_source = if let Some(path) = &req.pkg.archive_binary_path {
validate_shell_safe("archive binary path", path)?;
path.clone()
} else {
match archive_stem(req.asset_name) {
Some(stem) => format!("{stem}/{binary_name}"),
None => req.asset_name.to_string(),
}
};
Ok(format!(
"# Maintainer: pkgwatch (auto-generated — do not edit by hand,\n\
# edits are overwritten on the next update)\n\
pkgname='{name}'\n\
pkgver='{version}'\n\
pkgrel=1\n\
pkgdesc='{repo} release {version}, packaged by pkgwatch'\n\
arch=('x86_64')\n\
url='https://github.com/{repo}'\n\
license=('unknown')\n\
options=('!strip')\n\
source=('{asset}::{url}')\n\
sha256sums=('{sha256}')\n\
\n\
package() {{\n\
\x20 install -Dm755 \"${{srcdir}}/{install_source}\" \"${{pkgdir}}/usr/bin/{binary_name}\"\n\
}}\n",
name = req.pkg_name,
version = req.version,
repo = req.repo,
asset = req.asset_name,
url = req.download_url,
))
}
/// Matches `<prefix><anything>.pkg.tar.<compression>` — not hardcoded to
/// `.zst` specifically, since `PKGEXT` in makepkg.conf can be set to any
/// of pacman's supported compressions (`.xz`, `.gz`, `.bz2`, ...). This
/// box's default happens to be `.zst`, but guessing wrong would otherwise
/// report a false "makepkg failed" for a build that actually succeeded.
fn find_built_package(build_dir: &Path, pkg_name: &str, version: &str) -> Result<PathBuf> {
let prefix = format!("{pkg_name}-{version}-");
for entry in std::fs::read_dir(build_dir)? {
let path = entry?.path();
let Some(file_name) = path.file_name().and_then(|n| n.to_str()) else {
continue;
};
if file_name.starts_with(&prefix) && file_name.contains(".pkg.tar.") {
return Ok(path);
}
}
bail!(
"makepkg reported success but no {prefix}*.pkg.tar.* found in {}",
build_dir.display()
)
}
/// Strips a recognized archive extension, returning the resulting stem —
/// the directory name `makepkg` extracts a same-named tarball into, by
/// the convention every currently-tracked tarball-shaped package follows.
/// `None` means the asset is a bare binary download (no extraction).
fn archive_stem(asset_name: &str) -> Option<&str> {
ARCHIVE_EXTENSIONS
.iter()
.find_map(|ext| asset_name.strip_suffix(ext))
}
/// Rejects characters that are dangerous in *either* quoting style the
/// PKGBUILD template uses: a single quote breaks out of the single-quoted
/// fields (`pkgname`, `sha256sums`, ...); `$`, a backtick, or a backslash
/// are still live inside the double-quoted `install()` line, where
/// `asset_name` (via `install_source`) and `binary_name` end up embedded
/// so `${srcdir}`/`${pkgdir}` can expand. A single check covering both
/// contexts is safer than trying to remember which fields land in which
/// quoting style. See docs/SPEC.md > Architecture > Builder ("never unescaped
/// interpolation").
fn validate_shell_safe(field: &str, value: &str) -> Result<()> {
if value.contains(['\'', '\n', '$', '`', '\\']) {
bail!("{field} '{value}' contains an unsafe character for a generated PKGBUILD");
}
Ok(())
}
/// A pacman `pkgver` may only contain alphanumerics, `.`, `_`, `+` — no
/// hyphens (pacman reserves `-` as the pkgver/pkgrel separator in the
/// final package filename) and no shell metacharacters.
fn validate_pkgver(version: &str) -> Result<()> {
let valid = !version.is_empty()
&& version
.chars()
.all(|c| c.is_ascii_alphanumeric() || matches!(c, '.' | '_' | '+'));
if !valid {
bail!("'{version}' is not a valid pacman pkgver (only [A-Za-z0-9._+] allowed)");
}
Ok(())
}
/// A pacman package/binary name may only contain lowercase alphanumerics
/// plus `@ . _ + -`.
fn validate_pkgname(name: &str) -> Result<()> {
let valid = !name.is_empty()
&& name.chars().all(|c| {
c.is_ascii_lowercase() || c.is_ascii_digit() || matches!(c, '@' | '.' | '_' | '+' | '-')
});
if !valid {
bail!("'{name}' is not a valid pacman package name");
}
Ok(())
}
#[cfg(test)]
mod tests {
use super::*;
#[test]
fn archive_stem_strips_known_extensions() {
assert_eq!(
archive_stem("uv-x86_64-unknown-linux-gnu.tar.gz"),
Some("uv-x86_64-unknown-linux-gnu")
);
assert_eq!(archive_stem("thing.zip"), Some("thing"));
}
#[test]
fn archive_stem_none_for_bare_binary() {
assert_eq!(archive_stem("scaleway-cli_2.62.0_linux_amd64"), None);
}
#[test]
fn find_built_package_matches_default_zst_extension() {
let dir = tempfile::tempdir().unwrap();
std::fs::write(dir.path().join("uv-0.12.15-1-x86_64.pkg.tar.zst"), b"").unwrap();
let found = find_built_package(dir.path(), "uv", "0.12.15").unwrap();
assert_eq!(found, dir.path().join("uv-0.12.15-1-x86_64.pkg.tar.zst"));
}
#[test]
fn find_built_package_matches_non_default_pkgext() {
// A box with PKGEXT='.pkg.tar.xz' in makepkg.conf shouldn't report
// a false failure just because this crate's default assumption
// (.zst) doesn't match.
let dir = tempfile::tempdir().unwrap();
std::fs::write(dir.path().join("uv-0.12.15-1-x86_64.pkg.tar.xz"), b"").unwrap();
let found = find_built_package(dir.path(), "uv", "0.12.15").unwrap();
assert_eq!(found, dir.path().join("uv-0.12.15-1-x86_64.pkg.tar.xz"));
}
#[test]
fn find_built_package_ignores_non_matching_prefix() {
let dir = tempfile::tempdir().unwrap();
std::fs::write(dir.path().join("other-0.12.15-1-x86_64.pkg.tar.zst"), b"").unwrap();
assert!(find_built_package(dir.path(), "uv", "0.12.15").is_err());
}
#[test]
fn find_built_package_errors_with_clear_message_when_nothing_matches() {
let dir = tempfile::tempdir().unwrap();
let err = find_built_package(dir.path(), "uv", "0.12.15").unwrap_err();
assert!(err.to_string().contains("uv-0.12.15-"));
assert!(err.to_string().contains(".pkg.tar.*"));
}
#[test]
fn validate_pkgver_accepts_dotted_version() {
assert!(validate_pkgver("2.62.0").is_ok());
}
#[test]
fn validate_pkgver_rejects_hyphen() {
assert!(validate_pkgver("2.62.0-dbg1").is_err());
}
#[test]
fn validate_pkgver_rejects_shell_metacharacters() {
assert!(validate_pkgver("2.62.0; rm -rf /").is_err());
}
#[test]
fn validate_pkgver_rejects_empty() {
assert!(validate_pkgver("").is_err());
}
#[test]
fn validate_pkgname_accepts_hyphenated_name() {
assert!(validate_pkgname("scaleway-cli").is_ok());
}
#[test]
fn validate_pkgname_rejects_uppercase() {
assert!(validate_pkgname("Scaleway-CLI").is_err());
}
#[test]
fn validate_shell_safe_rejects_single_quote() {
assert!(validate_shell_safe("asset name", "thing'; touch pwned #.tar.gz").is_err());
}
#[test]
fn validate_shell_safe_rejects_newline() {
assert!(validate_shell_safe("download url", "https://example.com/a\nb").is_err());
}
#[test]
fn validate_shell_safe_rejects_dollar_sign() {
// asset_name lands inside a double-quoted string via
// install_source — $() command substitution is still live there
// even though single-quote breakout isn't.
assert!(validate_shell_safe("asset name", "thing$(touch pwned).tar.gz").is_err());
}
#[test]
fn validate_shell_safe_rejects_backtick() {
assert!(validate_shell_safe("asset name", "thing`touch pwned`.tar.gz").is_err());
}
#[test]
fn validate_shell_safe_rejects_backslash() {
assert!(validate_shell_safe("asset name", "thing\\$(touch pwned).tar.gz").is_err());
}
#[test]
fn validate_shell_safe_accepts_normal_url() {
assert!(validate_shell_safe("download url", "https://example.com/a/b.tar.gz").is_ok());
}
fn make_package(binary_name: Option<&str>) -> Package {
let toml_text = match binary_name {
Some(bin) => format!(
r#"
repo = "o/r"
asset_pattern = "x"
binary_name = "{bin}"
[verification]
method = "github-attestation"
"#
),
None => r#"
repo = "o/r"
asset_pattern = "x"
[verification]
method = "github-attestation"
"#
.to_string(),
};
toml::from_str(&toml_text).unwrap()
}
fn make_package_with_archive_binary_path(
binary_name: &str,
archive_binary_path: &str,
) -> Package {
let toml_text = format!(
r#"
repo = "o/r"
asset_pattern = "x"
binary_name = "{binary_name}"
archive_binary_path = "{archive_binary_path}"
[verification]
method = "github-attestation"
"#
);
toml::from_str(&toml_text).unwrap()
}
#[test]
fn build_rejects_unsafe_version() {
let pkg = make_package(None);
let dir = tempfile::tempdir().unwrap();
let artifact_path = dir.path().join("thing.tar.gz");
std::fs::write(&artifact_path, b"data").unwrap();
let req = BuildRequest {
pkg_name: "thing",
pkg: &pkg,
version: "1.0.0-dbg1",
repo: "o/r",
asset_name: "thing.tar.gz",
download_url: "https://example.com/thing.tar.gz",
artifact_path: &artifact_path,
};
let build_dir = dir.path().join("build");
assert!(build(&req, &build_dir).is_err());
}
#[test]
fn generate_pkgbuild_bare_binary_installs_under_binary_name_override() {
let pkg = make_package(Some("scw"));
let dir = tempfile::tempdir().unwrap();
let artifact_path = dir.path().join("scaleway-cli_2.62.0_linux_amd64");
std::fs::write(&artifact_path, b"binary-bytes").unwrap();
let expected_sha = hash::sha256_hex(b"binary-bytes");
let req = BuildRequest {
pkg_name: "scaleway-cli",
pkg: &pkg,
version: "2.62.0",
repo: "scaleway/scaleway-cli",
asset_name: "scaleway-cli_2.62.0_linux_amd64",
download_url: "https://github.com/scaleway/scaleway-cli/releases/download/v2.62.0/scaleway-cli_2.62.0_linux_amd64",
artifact_path: &artifact_path,
};
let pkgbuild = generate_pkgbuild(&req).unwrap();
assert!(pkgbuild.contains("pkgname='scaleway-cli'"));
assert!(pkgbuild.contains("pkgver='2.62.0'"));
assert!(pkgbuild.contains(&format!("sha256sums=('{expected_sha}')")));
// Bare binary (no archive extension) — installed straight from
// srcdir under the overridden binary name, not the pkgname.
assert!(pkgbuild.contains(
"install -Dm755 \"${srcdir}/scaleway-cli_2.62.0_linux_amd64\" \"${pkgdir}/usr/bin/scw\""
));
}
#[test]
fn generate_pkgbuild_tarball_installs_from_extracted_stem_dir() {
let pkg = make_package(None);
let dir = tempfile::tempdir().unwrap();
let artifact_path = dir.path().join("uv-x86_64-unknown-linux-gnu.tar.gz");
std::fs::write(&artifact_path, b"tarball-bytes").unwrap();
let req = BuildRequest {
pkg_name: "uv",
pkg: &pkg,
version: "0.12.15",
repo: "astral-sh/uv",
asset_name: "uv-x86_64-unknown-linux-gnu.tar.gz",
download_url: "https://github.com/astral-sh/uv/releases/download/0.12.15/uv-x86_64-unknown-linux-gnu.tar.gz",
artifact_path: &artifact_path,
};
let pkgbuild = generate_pkgbuild(&req).unwrap();
// No binary_name override — pkgname doubles as the binary name,
// and makepkg extracts the tarball into a same-named directory.
assert!(pkgbuild.contains(
"install -Dm755 \"${srcdir}/uv-x86_64-unknown-linux-gnu/uv\" \"${pkgdir}/usr/bin/uv\""
));
}
#[test]
fn generate_pkgbuild_flat_archive_installs_from_archive_binary_path_override() {
// claude-code's shape: a tarball with no wrapping directory, whose
// inner filename ("claude") doesn't match the package name
// ("claude-code") — neither existing shape (stem/binary_name, or
// bare-binary-no-archive) fits, hence the explicit override.
let pkg = make_package_with_archive_binary_path("claude-code", "claude");
let dir = tempfile::tempdir().unwrap();
let artifact_path = dir.path().join("claude-linux-x64.tar.gz");
std::fs::write(&artifact_path, b"tarball-bytes").unwrap();
let req = BuildRequest {
pkg_name: "claude-code",
pkg: &pkg,
version: "2.1.276",
repo: "anthropics/claude-code",
asset_name: "claude-linux-x64.tar.gz",
download_url: "https://github.com/anthropics/claude-code/releases/download/v2.1.276/claude-linux-x64.tar.gz",
artifact_path: &artifact_path,
};
let pkgbuild = generate_pkgbuild(&req).unwrap();
assert!(
pkgbuild
.contains("install -Dm755 \"${srcdir}/claude\" \"${pkgdir}/usr/bin/claude-code\"")
);
}
#[test]
fn generate_pkgbuild_rejects_archive_binary_path_with_command_substitution() {
let pkg = make_package_with_archive_binary_path("claude-code", "claude$(touch pwned)");
let dir = tempfile::tempdir().unwrap();
let artifact_path = dir.path().join("claude-linux-x64.tar.gz");
std::fs::write(&artifact_path, b"data").unwrap();
let req = BuildRequest {
pkg_name: "claude-code",
pkg: &pkg,
version: "2.1.276",
repo: "anthropics/claude-code",
asset_name: "claude-linux-x64.tar.gz",
download_url: "https://github.com/anthropics/claude-code/releases/download/v2.1.276/claude-linux-x64.tar.gz",
artifact_path: &artifact_path,
};
assert!(generate_pkgbuild(&req).is_err());
}
#[test]
fn generate_pkgbuild_rejects_download_url_with_single_quote() {
let pkg = make_package(None);
let dir = tempfile::tempdir().unwrap();
let artifact_path = dir.path().join("thing.tar.gz");
std::fs::write(&artifact_path, b"data").unwrap();
let req = BuildRequest {
pkg_name: "thing",
pkg: &pkg,
version: "1.0.0",
repo: "o/r",
asset_name: "thing.tar.gz",
download_url: "https://example.com/x'; touch pwned #.tar.gz",
artifact_path: &artifact_path,
};
assert!(generate_pkgbuild(&req).is_err());
}
#[test]
fn generate_pkgbuild_rejects_asset_name_with_command_substitution() {
// Regression test: asset_name feeds install_source, which is
// embedded in the double-quoted install() line, not a
// single-quoted field — a single-quote-only check would miss this.
let pkg = make_package(None);
let dir = tempfile::tempdir().unwrap();
let artifact_path = dir.path().join("thing.tar.gz");
std::fs::write(&artifact_path, b"data").unwrap();
let req = BuildRequest {
pkg_name: "thing",
pkg: &pkg,
version: "1.0.0",
repo: "o/r",
asset_name: "thing$(touch pwned).tar.gz",
download_url: "https://example.com/thing.tar.gz",
artifact_path: &artifact_path,
};
assert!(generate_pkgbuild(&req).is_err());
}
}

45
src/config.rs
View file

@ -1,3 +1,7 @@
//! Parses `packages.d/*.toml` into typed, in-memory `Package` records.
//! The only module that knows the TOML shape — everything downstream
//! works with `Package`/`Verification`/`SanityCheck`, never raw TOML.
use anyhow::{Context, Result}; use anyhow::{Context, Result};
use serde::Deserialize; use serde::Deserialize;
use std::collections::HashMap; use std::collections::HashMap;
@ -12,19 +16,54 @@ struct PackageFile {
pub struct Package { pub struct Package {
pub repo: String, pub repo: String,
/// Exact GitHub release asset name (still not a glob — see /// Exact GitHub release asset name (still not a glob — see
/// SPEC.md > Architecture > Fetcher), optionally containing a /// docs/SPEC.md > Architecture > Fetcher), optionally containing a
/// `{version}` placeholder for projects whose asset names embed the /// `{version}` placeholder for projects whose asset names embed the
/// version (e.g. `scaleway-cli_{version}_linux_amd64`). Substituted via /// version (e.g. `scaleway-cli_{version}_linux_amd64`). Substituted via
/// `checker::version_from_tag` before matching. /// `checker::version_from_tag` before matching.
pub asset_pattern: String, pub asset_pattern: String,
pub verification: Verification, pub verification: Verification,
/// Name of the executable inside the built package, if it differs from
/// the package name itself — e.g. scaleway-cli's pacman package is
/// named `scaleway-cli` but its real binary is `scw` (discovered by
/// checking the currently-installed extra package, not guessable from
/// the repo name). Defaults to the package name when omitted.
pub binary_name: Option<String>,
/// Explicit path to the binary inside the extracted archive, relative
/// to `srcdir`, for archive layouts that don't match the "extracts into
/// a directory named after the archive stem" convention `builder.rs`
/// otherwise assumes (e.g. claude-code's tarball extracts a bare
/// `claude` file with no wrapping directory, and that inner filename
/// doesn't match the package name either). Only meaningful when
/// `asset_pattern` names an archive; ignored for bare-binary downloads,
/// where the downloaded file *is* the source path already. Defaults to
/// the stem/`binary_name` convention when omitted.
pub archive_binary_path: Option<String>,
/// Post-build correctness check (not a security control — see
/// docs/SPEC.md > Verification trust tiers). Runs `command` against the
/// freshly built binary and confirms `version_regex`'s capture group
/// matches the version pkgwatch believes it just built.
pub sanity_check: Option<SanityCheck>,
}
impl Package {
/// The name of the executable inside the built package: `binary_name`
/// if the package declares one, else `pkg_name` itself.
pub fn binary_name<'a>(&'a self, pkg_name: &'a str) -> &'a str {
self.binary_name.as_deref().unwrap_or(pkg_name)
}
}
#[derive(Debug, Deserialize, Clone)]
pub struct SanityCheck {
pub command: String,
pub version_regex: String,
} }
#[derive(Debug, Deserialize, Clone)] #[derive(Debug, Deserialize, Clone)]
#[serde(tag = "method", rename_all = "kebab-case")] #[serde(tag = "method", rename_all = "kebab-case")]
pub enum Verification { pub enum Verification {
/// Tier 4: proves transport integrity only, not authorship. See /// Tier 4: proves transport integrity only, not authorship. See
/// SPEC.md > Verification trust tiers. `checksum_asset_pattern` may /// docs/SPEC.md > Verification trust tiers. `checksum_asset_pattern` may
/// also contain a `{version}` placeholder, same as `asset_pattern`. /// also contain a `{version}` placeholder, same as `asset_pattern`.
SameOriginSha256 { checksum_asset_pattern: String }, SameOriginSha256 { checksum_asset_pattern: String },
/// Tier 2: GitHub build-provenance attestation, verified via `gh /// Tier 2: GitHub build-provenance attestation, verified via `gh
@ -42,7 +81,7 @@ impl Verification {
} }
/// Loads every `*.toml` file in `dir` (the `packages.d/` layout from /// Loads every `*.toml` file in `dir` (the `packages.d/` layout from
/// SPEC.md > Scaling to many packages), keyed by package name. /// docs/SPEC.md > Scaling to many packages), keyed by package name.
pub fn load_packages_dir(dir: &Path) -> Result<Vec<(String, Package)>> { pub fn load_packages_dir(dir: &Path) -> Result<Vec<(String, Package)>> {
let mut out = Vec::new(); let mut out = Vec::new();
for entry in std::fs::read_dir(dir).with_context(|| format!("reading {}", dir.display()))? { for entry in std::fs::read_dir(dir).with_context(|| format!("reading {}", dir.display()))? {

30
src/fetcher.rs
View file

@ -1,3 +1,7 @@
//! Downloads a named GitHub release asset to a local path. The only
//! module that talks to the releases API for asset bytes — `checker` only
//! resolves version tags, never downloads.
use crate::github::GithubEndpoints; use crate::github::GithubEndpoints;
use anyhow::{Context, Result}; use anyhow::{Context, Result};
use serde::Deserialize; use serde::Deserialize;
@ -14,8 +18,18 @@ struct Asset {
browser_download_url: String, browser_download_url: String,
} }
/// A downloaded release asset: its local path plus the URL it came from,
/// the latter needed for the `source=` line of a generated PKGBUILD (see
/// `builder`) — `makepkg` uses it only as a fallback if the pre-seeded
/// local copy ever goes missing.
#[derive(Debug, Clone, PartialEq, Eq)]
pub struct DownloadedAsset {
pub path: PathBuf,
pub download_url: String,
}
/// Downloads the release asset named exactly `asset_name` for `repo`@`tag` /// Downloads the release asset named exactly `asset_name` for `repo`@`tag`
/// into `dest_dir`, returning the local path. /// into `dest_dir`, returning the local path and its origin URL.
pub fn download_asset( pub fn download_asset(
client: &reqwest::blocking::Client, client: &reqwest::blocking::Client,
endpoints: &GithubEndpoints, endpoints: &GithubEndpoints,
@ -23,7 +37,7 @@ pub fn download_asset(
tag: &str, tag: &str,
asset_name: &str, asset_name: &str,
dest_dir: &Path, dest_dir: &Path,
) -> Result<PathBuf> { ) -> Result<DownloadedAsset> {
let api_url = format!("{}/repos/{repo}/releases/tags/{tag}", endpoints.api); let api_url = format!("{}/repos/{repo}/releases/tags/{tag}", endpoints.api);
let release: Release = client let release: Release = client
.get(&api_url) .get(&api_url)
@ -46,7 +60,10 @@ pub fn download_asset(
.error_for_status()? .error_for_status()?
.bytes()?; .bytes()?;
std::fs::write(&dest_path, &bytes)?; std::fs::write(&dest_path, &bytes)?;
Ok(dest_path) Ok(DownloadedAsset {
path: dest_path,
download_url: asset.browser_download_url.clone(),
})
} }
#[cfg(test)] #[cfg(test)]
@ -77,7 +94,7 @@ mod tests {
let client = reqwest::blocking::Client::new(); let client = reqwest::blocking::Client::new();
let dest_dir = tempfile::tempdir().unwrap(); let dest_dir = tempfile::tempdir().unwrap();
let path = download_asset( let asset = download_asset(
&client, &client,
&endpoints, &endpoints,
"o/r", "o/r",
@ -87,8 +104,9 @@ mod tests {
) )
.unwrap(); .unwrap();
assert_eq!(path, dest_dir.path().join("thing.tar.gz")); assert_eq!(asset.path, dest_dir.path().join("thing.tar.gz"));
assert_eq!(std::fs::read(&path).unwrap(), b"artifact-bytes"); assert_eq!(asset.download_url, asset_url);
assert_eq!(std::fs::read(&asset.path).unwrap(), b"artifact-bytes");
} }
#[test] #[test]

81
src/hash.rs Normal file
View file

@ -0,0 +1,81 @@
//! Two functions, shared by two real callers (`verifier`, `builder`) —
//! not a general-purpose utils dump. See docs/ARCHITECTURE.md > "organize by
//! pipeline stage, not by layer" for why that distinction matters.
use anyhow::{Context, Result};
use sha2::{Digest, Sha256};
use std::io::Read;
use std::path::Path;
/// In-memory digest — test-only now that both real callers (`verifier`,
/// `builder`) hash a file already on disk via `sha256_hex_file` instead.
/// Kept for building expected hashes from in-memory test fixtures.
#[cfg(test)]
pub(crate) fn sha256_hex(data: &[u8]) -> String {
let mut hasher = Sha256::new();
hasher.update(data);
hex::encode(hasher.finalize())
}
/// Same digest as `sha256_hex(&std::fs::read(path)?)`, but streamed in
/// fixed-size chunks instead of reading the whole file into memory first —
/// downloaded release assets are tens of MB, and the file is already on
/// disk, so there's no reason to hold a second full copy in memory just to
/// hash it.
pub fn sha256_hex_file(path: &Path) -> Result<String> {
let mut file =
std::fs::File::open(path).with_context(|| format!("opening {}", path.display()))?;
let mut hasher = Sha256::new();
let mut buf = [0u8; 64 * 1024];
loop {
let n = file
.read(&mut buf)
.with_context(|| format!("reading {}", path.display()))?;
if n == 0 {
break;
}
hasher.update(&buf[..n]);
}
Ok(hex::encode(hasher.finalize()))
}
#[cfg(test)]
mod tests {
use super::*;
#[test]
fn matches_known_sha256() {
// printf 'hello world' | sha256sum
assert_eq!(
sha256_hex(b"hello world"),
"b94d27b9934d3e08a52e52d7da7dabfac484efe37a5380ee9088f7ace2efcde9"
);
}
#[test]
fn sha256_hex_file_matches_in_memory_digest() {
let dir = tempfile::tempdir().unwrap();
let path = dir.path().join("data.bin");
// Bigger than one read chunk, to actually exercise the loop.
let data = vec![0x5au8; 200 * 1024];
std::fs::write(&path, &data).unwrap();
assert_eq!(sha256_hex_file(&path).unwrap(), sha256_hex(&data));
}
#[test]
fn sha256_hex_file_matches_for_empty_file() {
let dir = tempfile::tempdir().unwrap();
let path = dir.path().join("empty.bin");
std::fs::write(&path, b"").unwrap();
assert_eq!(sha256_hex_file(&path).unwrap(), sha256_hex(b""));
}
#[test]
fn sha256_hex_file_errors_on_missing_file() {
let dir = tempfile::tempdir().unwrap();
let missing = dir.path().join("does-not-exist.bin");
assert!(sha256_hex_file(&missing).is_err());
}
}

103
src/main.rs
View file

@ -1,92 +1,33 @@
//! Entry point: parses `argv` and dispatches to `pipeline`. Nothing here
//! makes a network/subprocess call or contains a decision worth a test —
//! see docs/ARCHITECTURE.md > "main is a dispatcher, not the program."
mod builder;
mod checker; mod checker;
mod config; mod config;
mod fetcher; mod fetcher;
mod github; mod github;
mod hash;
mod pipeline;
mod publisher;
mod sanity;
mod state; mod state;
#[cfg(test)]
mod test_support;
mod verifier; mod verifier;
use anyhow::Result; use anyhow::{Result, bail};
use github::GithubEndpoints;
use std::path::Path;
/// First iteration: check -> fetch -> verify -> report, for whatever is /// check -> fetch -> verify -> build -> sanity-check -> publish, for
/// in packages.d/. No build/publish step yet (see SPEC.md > Status). /// whatever is in packages.d/. Tier 1-3 passes auto-publish; tier 4-6
/// passes queue for `pkgwatch review`. See docs/SPEC.md > Architecture for what
/// each stage does, and docs/ARCHITECTURE.md for how the code implementing it
/// is organized.
fn main() -> Result<()> { fn main() -> Result<()> {
let client = reqwest::blocking::Client::builder() let args: Vec<String> = std::env::args().skip(1).collect();
.user_agent("pkgwatch/0.1 (PoC; https://code.austinschaefer.com)") match args.first().map(String::as_str) {
.build()?; None => pipeline::run_check(),
let endpoints = GithubEndpoints::default(); Some("review") => pipeline::run_review(&args[1..]),
Some(other) => bail!("unknown subcommand '{other}' (expected: review)"),
let packages_dir = Path::new("packages.d");
let state_dir = Path::new("state");
let work_dir = Path::new("work");
let packages = config::load_packages_dir(packages_dir)?;
if packages.is_empty() {
println!("no packages configured under {}/", packages_dir.display());
return Ok(());
}
for (name, pkg) in packages {
println!("== {name} ({}) ==", pkg.repo);
let latest = checker::latest_github_release(&client, &endpoints, &pkg.repo)?;
let last_seen = state::load_last_version(state_dir, &name);
if last_seen.as_deref() == Some(latest.as_str()) {
println!(" up to date at {latest}");
continue;
}
println!(" new version detected: {latest} (previously: {last_seen:?})");
let dest_dir = work_dir.join(&name).join(&latest);
let asset_name = pkg
.asset_pattern
.replace("{version}", checker::version_from_tag(&latest));
let artifact_path = fetcher::download_asset(
&client,
&endpoints,
&pkg.repo,
&latest,
&asset_name,
&dest_dir,
)?;
println!(" fetched {}", artifact_path.display());
let result = verifier::verify(
&client,
&endpoints,
&pkg.verification,
&pkg.repo,
&latest,
&artifact_path,
&dest_dir,
)?;
println!(
" verification (tier {}): {} — {}",
result.tier,
if result.passed { "PASS" } else { "FAIL" },
result.justification
);
match (result.tier, result.passed) {
(1..=3, true) => {
println!(" tier 1-3 pass: would auto-build + publish (not yet implemented)");
state::save_last_version(state_dir, &name, &latest)?;
}
(_, true) => {
println!(" tier 4-6 pass: flagging for human review, not auto-publishing");
println!(
" (review-queue persistence not yet implemented — this is where it plugs in)"
);
}
(_, false) => {
println!(" verification failed — not publishing, not updating state");
} }
} }
}
Ok(())
}

436
src/pipeline.rs Normal file
View file

@ -0,0 +1,436 @@
//! Orchestrates one run of check -> fetch -> verify -> build ->
//! sanity-check -> publish across every configured package, plus the
//! `review` subcommand for tier 4-6 approvals. The only module that calls
//! more than one other pipeline-stage module — see docs/ARCHITECTURE.md > "main
//! is a dispatcher, not the program" for why this lives here and not in
//! `main.rs`.
use crate::builder;
use crate::checker;
use crate::config::{self, Package};
use crate::fetcher::{self, DownloadedAsset};
use crate::github::GithubEndpoints;
use crate::publisher;
use crate::sanity;
use crate::state;
use crate::verifier::{self, VerificationResult};
use anyhow::{Context, Result, bail};
use std::path::{Path, PathBuf};
const PACKAGES_DIR: &str = "packages.d";
const STATE_DIR: &str = "state";
const WORK_DIR: &str = "work";
/// Not a repo pkgwatch invents: this is the existing, already-registered
/// local pacman repo on this box (see `[custom]` in /etc/pacman.conf and
/// its `Server = file://...` line). pkgwatch adds packages to it; it does
/// not create the repo or touch pacman.conf.
const CUSTOM_REPO_NAME: &str = "custom";
const CUSTOM_REPO_SUBPATH: &str = ".local/share/pacman/custom";
fn build_client() -> Result<reqwest::blocking::Client> {
Ok(reqwest::blocking::Client::builder()
.user_agent("pkgwatch/0.1 (PoC; https://code.austinschaefer.com)")
.build()?)
}
fn custom_repo_dir() -> Result<PathBuf> {
// Override for testing against a scratch repo instead of the real one
// at $HOME/.local/share/pacman/custom.
if let Ok(dir) = std::env::var("PKGWATCH_REPO_DIR") {
return Ok(PathBuf::from(dir));
}
let home = std::env::var("HOME").context("HOME is not set")?;
Ok(Path::new(&home).join(CUSTOM_REPO_SUBPATH))
}
pub fn run_check() -> Result<()> {
let client = build_client()?;
let endpoints = GithubEndpoints::default();
let packages_dir = Path::new(PACKAGES_DIR);
let state_dir = Path::new(STATE_DIR);
let work_dir = Path::new(WORK_DIR);
let packages = config::load_packages_dir(packages_dir)?;
if packages.is_empty() {
println!("no packages configured under {}/", packages_dir.display());
return Ok(());
}
let mut any_failed = false;
for (name, pkg) in &packages {
println!("== {name} ({}) ==", pkg.repo);
if let Err(err) = process_package(&client, &endpoints, state_dir, work_dir, name, pkg) {
eprintln!(" error: {err:#}");
any_failed = true;
}
}
if any_failed {
bail!("one or more packages failed — see errors above");
}
Ok(())
}
/// What to do about a package after verification, derived purely from the
/// verification outcome and whether this exact version is already queued
/// for review — no I/O. See docs/ARCHITECTURE.md > "separate pure decision
/// logic from I/O."
#[derive(Debug, Clone, Copy, PartialEq, Eq)]
enum TierAction {
/// Verification failed outright — don't build, don't touch state.
VerificationFailed,
/// Tier 1-3: safe to auto-build and publish immediately.
Publish,
/// Tier 4-6, and this exact version was already flagged on an earlier
/// run — nothing new to report.
StillPending,
/// Tier 4-6, and this version hasn't been flagged yet.
NewlyPending,
}
fn decide_tier_action(tier: u8, passed: bool, already_pending_this_version: bool) -> TierAction {
if !passed {
return TierAction::VerificationFailed;
}
if tier <= 3 {
return TierAction::Publish;
}
if already_pending_this_version {
TierAction::StillPending
} else {
TierAction::NewlyPending
}
}
fn process_package(
client: &reqwest::blocking::Client,
endpoints: &GithubEndpoints,
state_dir: &Path,
work_dir: &Path,
name: &str,
pkg: &Package,
) -> Result<()> {
let latest = checker::latest_github_release(client, endpoints, &pkg.repo)?;
let last_seen = state::load_last_version(state_dir, name);
if last_seen.as_deref() == Some(latest.as_str()) {
println!(" up to date at {latest}");
return Ok(());
}
println!(" new version detected: {latest} (previously: {last_seen:?})");
let fetched = fetch_and_verify(client, endpoints, work_dir, name, pkg, &latest)?;
println!(" fetched {}", fetched.asset.path.display());
println!(
" verification (tier {}): {} — {}",
fetched.verification.tier,
if fetched.verification.passed {
"PASS"
} else {
"FAIL"
},
fetched.verification.justification
);
let previously_pending = state::load_pending_version(state_dir, name);
let already_pending = previously_pending.as_deref() == Some(latest.as_str());
match decide_tier_action(
fetched.verification.tier,
fetched.verification.passed,
already_pending,
) {
// Bail rather than just print-and-return: a verification failure
// is exactly the kind of event a monitoring setup (systemd
// OnFailure=, cron mail-on-error) needs a non-zero exit to catch —
// see run_check, which treats an Err here as a failed package.
TierAction::VerificationFailed => {
bail!("verification failed — not publishing, not updating state");
}
TierAction::Publish => {
println!(" tier 1-3 pass: building + publishing");
build_and_publish(name, pkg, &latest, &fetched)?;
state::save_last_version(state_dir, name, &latest)?;
state::clear_pending_version(state_dir, name)?;
println!(" published {name} {latest}");
}
TierAction::StillPending => {
println!(" tier 4-6 pass: still pending review (`pkgwatch review` to see it)");
}
TierAction::NewlyPending => {
state::save_pending_version(state_dir, name, &latest)?;
match previously_pending {
Some(superseded) => println!(
" tier 4-6 pass: flagged for human review, superseding still-unreviewed {superseded} (`pkgwatch review` to approve {latest})"
),
None => println!(
" tier 4-6 pass: flagged for human review (`pkgwatch review` to approve)"
),
}
}
}
Ok(())
}
struct FetchVerifyResult {
version: String,
asset_name: String,
dest_dir: PathBuf,
asset: DownloadedAsset,
verification: VerificationResult,
}
/// Shared by the normal check loop (tier 1-3 auto-path) and `pkgwatch
/// review --approve` (which re-verifies before publishing rather than
/// trusting a possibly-stale flag from an earlier run).
fn fetch_and_verify(
client: &reqwest::blocking::Client,
endpoints: &GithubEndpoints,
work_dir: &Path,
name: &str,
pkg: &Package,
tag: &str,
) -> Result<FetchVerifyResult> {
let version = checker::version_from_tag(tag).to_string();
let asset_name = pkg.asset_pattern.replace("{version}", &version);
let dest_dir = work_dir.join(name).join(tag);
let asset = fetcher::download_asset(client, endpoints, &pkg.repo, tag, &asset_name, &dest_dir)?;
let verification = verifier::verify(
client,
endpoints,
&pkg.verification,
&pkg.repo,
tag,
&asset.path,
&dest_dir,
)?;
Ok(FetchVerifyResult {
version,
asset_name,
dest_dir,
asset,
verification,
})
}
fn build_and_publish(
name: &str,
pkg: &Package,
tag: &str,
fetched: &FetchVerifyResult,
) -> Result<()> {
// Fail fast if the repo isn't registered in pacman.conf, before
// spending several seconds on a makepkg build that would otherwise
// succeed and then publish somewhere pacman never syncs from.
let repo_dir = custom_repo_dir()?;
publisher::ensure_registered(CUSTOM_REPO_NAME, &repo_dir)?;
let build_dir = fetched.dest_dir.join("build");
let req = builder::BuildRequest {
pkg_name: name,
pkg,
version: &fetched.version,
repo: &pkg.repo,
asset_name: &fetched.asset_name,
download_url: &fetched.asset.download_url,
artifact_path: &fetched.asset.path,
};
let built = builder::build(&req, &build_dir)?;
println!(" built {}", built.package_path.display());
if let Some(check) = &pkg.sanity_check {
let bin_dir = built.pkgdir.join("usr/bin");
sanity::run(check, &bin_dir, &fetched.version)
.with_context(|| format!("sanity check for {name} {tag}"))?;
println!(" sanity check passed");
}
let published = publisher::publish(&built.package_path, &repo_dir, CUSTOM_REPO_NAME)?;
println!(
" added to {} repo: {}",
CUSTOM_REPO_NAME,
published.display()
);
println!(
" not installed automatically — run `sudo pacman -Syu` (or `sudo pacman -S {name}`) to pick it up"
);
Ok(())
}
pub fn run_review(args: &[String]) -> Result<()> {
let packages_dir = Path::new(PACKAGES_DIR);
let state_dir = Path::new(STATE_DIR);
let work_dir = Path::new(WORK_DIR);
let packages = config::load_packages_dir(packages_dir)?;
match args {
[] => {
let mut any = false;
for (name, _) in &packages {
if let Some(pending) = state::load_pending_version(state_dir, name) {
println!(
"{name}: {pending} pending review (run `pkgwatch review {name} --approve`)"
);
any = true;
}
}
if !any {
println!("no packages pending review");
}
Ok(())
}
[name, flag] if flag == "--approve" => {
let (_, pkg) = packages.iter().find(|(n, _)| n == name).with_context(|| {
format!("no package named '{name}' in {}/", packages_dir.display())
})?;
let tag = state::load_pending_version(state_dir, name)
.with_context(|| format!("'{name}' has no pending review"))?;
approve(state_dir, work_dir, name, pkg, &tag)
}
_ => bail!("usage: pkgwatch review [<name> --approve]"),
}
}
fn approve(state_dir: &Path, work_dir: &Path, name: &str, pkg: &Package, tag: &str) -> Result<()> {
let client = build_client()?;
let endpoints = GithubEndpoints::default();
// Re-verify rather than trusting the earlier flag: the artifact at
// this tag could in principle have changed since it was queued.
let fetched = fetch_and_verify(&client, &endpoints, work_dir, name, pkg, tag)?;
if !fetched.verification.passed {
bail!(
"re-verification failed on approve: {}",
fetched.verification.justification
);
}
println!(
"re-verified (tier {}): {}",
fetched.verification.tier, fetched.verification.justification
);
build_and_publish(name, pkg, tag, &fetched)?;
state::save_last_version(state_dir, name, tag)?;
state::clear_pending_version(state_dir, name)?;
println!("approved and published {name} {tag}");
Ok(())
}
#[cfg(test)]
mod tests {
use super::*;
#[test]
fn decide_tier_action_failed_verification_overrides_everything() {
assert_eq!(
decide_tier_action(2, false, false),
TierAction::VerificationFailed
);
assert_eq!(
decide_tier_action(4, false, true),
TierAction::VerificationFailed
);
}
#[test]
fn decide_tier_action_tier_1_to_3_publishes() {
assert_eq!(decide_tier_action(1, true, false), TierAction::Publish);
assert_eq!(decide_tier_action(2, true, false), TierAction::Publish);
assert_eq!(decide_tier_action(3, true, true), TierAction::Publish);
}
#[test]
fn decide_tier_action_tier_4_to_6_newly_pending_when_not_seen_before() {
assert_eq!(decide_tier_action(4, true, false), TierAction::NewlyPending);
assert_eq!(decide_tier_action(6, true, false), TierAction::NewlyPending);
}
#[test]
fn decide_tier_action_tier_4_to_6_still_pending_when_already_flagged() {
assert_eq!(decide_tier_action(4, true, true), TierAction::StillPending);
}
/// Regression test for the exit-code gap this PR fixes: a verification
/// failure previously returned `Ok(())` from `process_package`, so
/// `run_check` never counted it as a failure and the process exited 0
/// even though the single most security-relevant check had failed.
/// Exercises the full check -> fetch -> verify path against a mocked
/// GitHub (no real network), stopping before any build/publish step
/// since verification failure returns before reaching those.
#[test]
fn process_package_returns_err_on_verification_failure() {
let mut server = mockito::Server::new();
let endpoints = GithubEndpoints {
web: server.url(),
api: server.url(),
};
let feed = format!(
r#"<feed><link rel="alternate" href="{}/o/r/releases/tag/v1.0.0"/></feed>"#,
server.url()
);
let _atom = server
.mock("GET", "/o/r/releases.atom")
.with_status(200)
.with_body(feed)
.create();
let asset_url = format!("{}/download/thing.tar.gz", server.url());
let sums_url = format!("{}/download/SHA256SUMS", server.url());
let release_body = format!(
r#"{{"assets": [
{{"name": "thing.tar.gz", "browser_download_url": "{asset_url}"}},
{{"name": "SHA256SUMS", "browser_download_url": "{sums_url}"}}
]}}"#
);
let _release = server
.mock("GET", "/repos/o/r/releases/tags/v1.0.0")
.with_status(200)
.with_body(release_body)
.create();
let _asset = server
.mock("GET", "/download/thing.tar.gz")
.with_status(200)
.with_body(b"artifact-bytes".as_slice())
.create();
// Wrong hash for "artifact-bytes" — forces a verification failure.
let _sums = server
.mock("GET", "/download/SHA256SUMS")
.with_status(200)
.with_body(
"0000000000000000000000000000000000000000000000000000000000000000 thing.tar.gz\n",
)
.create();
let pkg: Package = toml::from_str(
r#"
repo = "o/r"
asset_pattern = "thing.tar.gz"
[verification]
method = "same-origin-sha256"
checksum_asset_pattern = "SHA256SUMS"
"#,
)
.unwrap();
let client = reqwest::blocking::Client::new();
let state_dir = tempfile::tempdir().unwrap();
let work_dir = tempfile::tempdir().unwrap();
let err = process_package(
&client,
&endpoints,
state_dir.path(),
work_dir.path(),
"thing",
&pkg,
)
.unwrap_err();
assert!(err.to_string().contains("verification failed"));
// Neither published nor queued for review — a failed verification
// shouldn't leave any trace in state.
assert_eq!(state::load_last_version(state_dir.path(), "thing"), None);
assert_eq!(state::load_pending_version(state_dir.path(), "thing"), None);
}
}

200
src/publisher.rs Normal file
View file

@ -0,0 +1,200 @@
//! Gets a built package into the local pacman repo: copies it in, runs
//! `repo-add`, and checks the repo is actually registered in
//! `/etc/pacman.conf` first. The only module that touches the repo
//! directory or pacman.conf.
use anyhow::{Context, Result, bail};
use std::path::{Path, PathBuf};
use std::process::Command;
/// Pacman's system-wide config — hardcoded like the rest of this tool's
/// Arch/Manjaro-specific assumptions (see docs/SPEC.md > Scope).
const PACMAN_CONF: &str = "/etc/pacman.conf";
/// Copies the built package into `repo_dir` and runs `repo-add` against
/// `<repo_name>.db.tar.gz` there. Creates `repo_dir` if it doesn't exist
/// yet — `repo-add` itself creates the database file on its first run, so
/// everything filesystem-side is self-healing.
///
/// What is *not* self-healing, and can't safely be: registering
/// `repo_name` in `/etc/pacman.conf` (see `ensure_registered`) — that
/// needs root, which this process doesn't have and shouldn't grab for
/// itself. Getting a published version onto the running system is a
/// separate, deliberate `pacman -Syu`/`pacman -S` step too, also left to
/// the operator.
pub fn publish(package_path: &Path, repo_dir: &Path, repo_name: &str) -> Result<PathBuf> {
publish_with(Path::new("repo-add"), package_path, repo_dir, repo_name)
}
/// `repo_add_bin` is injectable so tests can point it at a stub script
/// instead of the real `repo-add` (or a mutated global `PATH`, which would
/// race with `cargo test`'s parallel test threads).
fn publish_with(
repo_add_bin: &Path,
package_path: &Path,
repo_dir: &Path,
repo_name: &str,
) -> Result<PathBuf> {
std::fs::create_dir_all(repo_dir)
.with_context(|| format!("creating repo dir {}", repo_dir.display()))?;
let file_name = package_path
.file_name()
.context("built package path has no filename")?;
let dest = repo_dir.join(file_name);
std::fs::copy(package_path, &dest)
.with_context(|| format!("copying {} to {}", package_path.display(), dest.display()))?;
let db_path = repo_dir.join(format!("{repo_name}.db.tar.gz"));
let status = Command::new(repo_add_bin)
.arg(&db_path)
.arg(&dest)
.status()
.with_context(|| {
format!(
"running {} (is pacman-contrib installed?)",
repo_add_bin.display()
)
})?;
if !status.success() {
bail!("repo-add failed for {}", dest.display());
}
Ok(dest)
}
/// Verifies `repo_name` is registered as an active `[section]` in
/// `/etc/pacman.conf`, so a build isn't wasted on a repo pacman will never
/// actually sync from. Call this before `publish` — ideally before even
/// starting the build, so a missing repo fails fast instead of after
/// several seconds of `makepkg` work.
///
/// Doesn't check that the section's `Server =`/`Include =` line points at
/// `repo_dir` specifically — just that a repo by this name exists at all.
/// A same-named repo pointed somewhere else is a rare, easily-diagnosed
/// misconfiguration, not worth the parsing complexity to catch here.
pub fn ensure_registered(repo_name: &str, repo_dir: &Path) -> Result<()> {
ensure_registered_at(Path::new(PACMAN_CONF), repo_name, repo_dir)
}
fn ensure_registered_at(pacman_conf: &Path, repo_name: &str, repo_dir: &Path) -> Result<()> {
let conf = std::fs::read_to_string(pacman_conf)
.with_context(|| format!("reading {}", pacman_conf.display()))?;
let header = format!("[{repo_name}]");
let registered = conf.lines().map(str::trim).any(|line| line == header);
if !registered {
bail!(
"'{repo_name}' is not registered in {} — add this once, as root, then re-run:\n\n\
[{repo_name}]\n\
SigLevel = Optional TrustAll\n\
Server = file://{}\n",
pacman_conf.display(),
repo_dir.display()
);
}
Ok(())
}
#[cfg(test)]
mod tests {
use super::*;
use crate::test_support::write_executable_script;
#[test]
fn publish_copies_package_and_invokes_repo_add() {
let stub_dir = tempfile::tempdir().unwrap();
let log_path = stub_dir.path().join("invoked_with.txt");
let repo_add = write_executable_script(
stub_dir.path(),
"fake-repo-add",
&format!("echo \"$@\" > {}", log_path.display()),
);
let src_dir = tempfile::tempdir().unwrap();
let package_path = src_dir.path().join("thing-1.0.0-1-x86_64.pkg.tar.zst");
std::fs::write(&package_path, b"pkg-bytes").unwrap();
let repo_dir = tempfile::tempdir().unwrap();
let dest = publish_with(&repo_add, &package_path, repo_dir.path(), "custom").unwrap();
assert_eq!(
dest,
repo_dir.path().join("thing-1.0.0-1-x86_64.pkg.tar.zst")
);
assert_eq!(std::fs::read(&dest).unwrap(), b"pkg-bytes");
let invoked_with = std::fs::read_to_string(&log_path).unwrap();
assert!(invoked_with.contains("custom.db.tar.gz"));
assert!(invoked_with.contains("thing-1.0.0-1-x86_64.pkg.tar.zst"));
}
#[test]
fn publish_errors_when_repo_add_fails() {
let stub_dir = tempfile::tempdir().unwrap();
let repo_add = write_executable_script(stub_dir.path(), "fake-repo-add-fail", "exit 1");
let src_dir = tempfile::tempdir().unwrap();
let package_path = src_dir.path().join("thing-1.0.0-1-x86_64.pkg.tar.zst");
std::fs::write(&package_path, b"pkg-bytes").unwrap();
let repo_dir = tempfile::tempdir().unwrap();
let err = publish_with(&repo_add, &package_path, repo_dir.path(), "custom").unwrap_err();
assert!(err.to_string().contains("repo-add failed"));
}
#[test]
fn publish_creates_repo_dir_if_missing() {
let stub_dir = tempfile::tempdir().unwrap();
let repo_add = write_executable_script(stub_dir.path(), "fake-repo-add-ok", "exit 0");
let src_dir = tempfile::tempdir().unwrap();
let package_path = src_dir.path().join("thing-1.0.0-1-x86_64.pkg.tar.zst");
std::fs::write(&package_path, b"pkg-bytes").unwrap();
let parent = tempfile::tempdir().unwrap();
let repo_dir = parent.path().join("nested/repo");
publish_with(&repo_add, &package_path, &repo_dir, "custom").unwrap();
assert!(repo_dir.join("thing-1.0.0-1-x86_64.pkg.tar.zst").exists());
}
fn write_pacman_conf(dir: &Path, contents: &str) -> PathBuf {
let path = dir.join("pacman.conf");
std::fs::write(&path, contents).unwrap();
path
}
#[test]
fn ensure_registered_passes_when_section_present() {
let dir = tempfile::tempdir().unwrap();
let conf = write_pacman_conf(
dir.path(),
"[options]\nArchitecture = auto\n\n[extra]\nInclude = /etc/pacman.d/mirrorlist\n\n[custom]\nSigLevel = Optional TrustAll\nServer = file:///home/austin/.local/share/pacman/custom\n",
);
assert!(ensure_registered_at(&conf, "custom", Path::new("/repo")).is_ok());
}
#[test]
fn ensure_registered_fails_when_section_missing() {
let dir = tempfile::tempdir().unwrap();
let conf = write_pacman_conf(dir.path(), "[options]\nArchitecture = auto\n\n[extra]\n");
let err = ensure_registered_at(&conf, "custom", Path::new("/repo")).unwrap_err();
let msg = err.to_string();
assert!(msg.contains("not registered"));
assert!(msg.contains("[custom]"));
assert!(msg.contains("/repo"));
}
#[test]
fn ensure_registered_does_not_match_substring_of_another_section() {
// "custom" must match the whole section header, not just appear
// as a substring of e.g. "[custom-extra]".
let dir = tempfile::tempdir().unwrap();
let conf = write_pacman_conf(dir.path(), "[custom-extra]\nServer = file:///elsewhere\n");
assert!(ensure_registered_at(&conf, "custom", Path::new("/repo")).is_err());
}
#[test]
fn ensure_registered_errors_when_pacman_conf_missing() {
let dir = tempfile::tempdir().unwrap();
let missing = dir.path().join("does-not-exist.conf");
assert!(ensure_registered_at(&missing, "custom", Path::new("/repo")).is_err());
}
}

122
src/sanity.rs Normal file
View file

@ -0,0 +1,122 @@
//! Post-build correctness check: runs the freshly built binary and
//! confirms it reports the version pkgwatch believes it just built. Not a
//! security control — see docs/SPEC.md > Verification trust tiers.
use crate::config::SanityCheck;
use anyhow::{Context, Result, bail};
use regex::Regex;
use std::path::Path;
use std::process::Command;
/// Runs `check.command` with `pkg_bin_dir` prepended to `PATH`, so it
/// exercises the binary pkgwatch just built (still sitting in makepkg's
/// package staging directory, not installed system-wide) rather than
/// whatever's already on the system. Confirms `check.version_regex`'s
/// capture group matches `expected_version`.
///
/// Correctness check only, not a security control — see docs/SPEC.md >
/// Verification trust tiers. Catches checker bugs and mangled/wrong-asset
/// downloads, not malicious releases.
pub fn run(check: &SanityCheck, pkg_bin_dir: &Path, expected_version: &str) -> Result<()> {
let path_env = format!(
"{}:{}",
pkg_bin_dir.display(),
std::env::var("PATH").unwrap_or_default()
);
let output = Command::new("sh")
.arg("-c")
.arg(&check.command)
.env("PATH", path_env)
.output()
.with_context(|| format!("running sanity check command '{}'", check.command))?;
if !output.status.success() {
bail!(
"sanity check command '{}' exited with {}: {}",
check.command,
output.status,
String::from_utf8_lossy(&output.stderr).trim()
);
}
let combined = format!(
"{}{}",
String::from_utf8_lossy(&output.stdout),
String::from_utf8_lossy(&output.stderr)
);
let re = Regex::new(&check.version_regex)
.with_context(|| format!("invalid version_regex '{}'", check.version_regex))?;
let found = re
.captures(&combined)
.and_then(|caps| caps.get(1))
.with_context(|| {
format!(
"version_regex '{}' did not match sanity check output: {combined:?}",
check.version_regex
)
})?
.as_str();
if found != expected_version {
bail!(
"sanity check reported version '{found}', pkgwatch built '{expected_version}' — mismatch"
);
}
Ok(())
}
#[cfg(test)]
mod tests {
use super::*;
use crate::test_support::write_executable_script as write_fake_binary;
#[test]
fn run_passes_when_reported_version_matches() {
let dir = tempfile::tempdir().unwrap();
write_fake_binary(dir.path(), "uv", "echo 'uv 0.12.15 (abc 2026-09-01)'");
let check = SanityCheck {
command: "uv --version".to_string(),
version_regex: r"uv (\d+\.\d+\.\d+)".to_string(),
};
assert!(run(&check, dir.path(), "0.12.15").is_ok());
}
#[test]
fn run_fails_when_reported_version_differs() {
let dir = tempfile::tempdir().unwrap();
write_fake_binary(dir.path(), "uv", "echo 'uv 0.12.14 (abc 2026-08-01)'");
let check = SanityCheck {
command: "uv --version".to_string(),
version_regex: r"uv (\d+\.\d+\.\d+)".to_string(),
};
let err = run(&check, dir.path(), "0.12.15").unwrap_err();
assert!(err.to_string().contains("mismatch"));
}
#[test]
fn run_fails_when_command_exits_nonzero() {
let dir = tempfile::tempdir().unwrap();
write_fake_binary(dir.path(), "uv", "exit 1");
let check = SanityCheck {
command: "uv --version".to_string(),
version_regex: r"uv (\d+\.\d+\.\d+)".to_string(),
};
let err = run(&check, dir.path(), "0.12.15").unwrap_err();
assert!(err.to_string().contains("exited with"));
}
#[test]
fn run_fails_when_output_does_not_match_regex() {
let dir = tempfile::tempdir().unwrap();
write_fake_binary(dir.path(), "uv", "echo 'not a version'");
let check = SanityCheck {
command: "uv --version".to_string(),
version_regex: r"uv (\d+\.\d+\.\d+)".to_string(),
};
let err = run(&check, dir.path(), "0.12.15").unwrap_err();
assert!(err.to_string().contains("did not match"));
}
}

78
src/state.rs
View file

@ -1,10 +1,14 @@
//! Persists two independent per-package facts as plain files: the last
//! published version, and any version currently pending human review.
//! The only module that touches `state/` on disk.
use anyhow::Result; use anyhow::Result;
use std::path::Path; use std::path::Path;
/// Last-known-published version per package, so re-runs don't re-flag a /// Last-known-published version per package, so re-runs don't re-flag a
/// version already handled. Deliberately just one file per package for /// version already handled. Deliberately just one file per package for
/// now — this is where a real review-queue persistence layer plugs in /// now — this is where a real review-queue persistence layer plugs in
/// later (see SPEC.md > Architecture > Reviewer queue). /// later (see docs/SPEC.md > Architecture > Reviewer queue).
pub fn load_last_version(state_dir: &Path, name: &str) -> Option<String> { pub fn load_last_version(state_dir: &Path, name: &str) -> Option<String> {
std::fs::read_to_string(state_dir.join(format!("{name}.version"))) std::fs::read_to_string(state_dir.join(format!("{name}.version")))
.ok() .ok()
@ -17,6 +21,33 @@ pub fn save_last_version(state_dir: &Path, name: &str, version: &str) -> Result<
Ok(()) Ok(())
} }
/// Tag currently awaiting human review for a tier 4-6 package (see
/// docs/SPEC.md > Architecture > Reviewer queue), if any. Separate from
/// `load_last_version`/`save_last_version`: approving a review doesn't
/// mean future versions auto-publish, so the two must be tracked
/// independently.
pub fn load_pending_version(state_dir: &Path, name: &str) -> Option<String> {
std::fs::read_to_string(state_dir.join(format!("{name}.pending")))
.ok()
.map(|s| s.trim().to_string())
}
pub fn save_pending_version(state_dir: &Path, name: &str, version: &str) -> Result<()> {
std::fs::create_dir_all(state_dir)?;
std::fs::write(state_dir.join(format!("{name}.pending")), version)?;
Ok(())
}
/// Clears a pending review, e.g. once it's been approved and published.
/// Not an error if there was nothing pending.
pub fn clear_pending_version(state_dir: &Path, name: &str) -> Result<()> {
match std::fs::remove_file(state_dir.join(format!("{name}.pending"))) {
Ok(()) => Ok(()),
Err(e) if e.kind() == std::io::ErrorKind::NotFound => Ok(()),
Err(e) => Err(e.into()),
}
}
#[cfg(test)] #[cfg(test)]
mod tests { mod tests {
use super::*; use super::*;
@ -57,4 +88,49 @@ mod tests {
Some("0.12.15".to_string()) Some("0.12.15".to_string())
); );
} }
#[test]
fn load_pending_version_missing_file_returns_none() {
let dir = tempfile::tempdir().unwrap();
assert_eq!(load_pending_version(dir.path(), "scaleway-cli"), None);
}
#[test]
fn save_then_load_pending_roundtrips() {
let dir = tempfile::tempdir().unwrap();
save_pending_version(dir.path(), "scaleway-cli", "v2.62.0").unwrap();
assert_eq!(
load_pending_version(dir.path(), "scaleway-cli"),
Some("v2.62.0".to_string())
);
}
#[test]
fn clear_pending_version_removes_it() {
let dir = tempfile::tempdir().unwrap();
save_pending_version(dir.path(), "scaleway-cli", "v2.62.0").unwrap();
clear_pending_version(dir.path(), "scaleway-cli").unwrap();
assert_eq!(load_pending_version(dir.path(), "scaleway-cli"), None);
}
#[test]
fn clear_pending_version_is_a_noop_when_nothing_pending() {
let dir = tempfile::tempdir().unwrap();
assert!(clear_pending_version(dir.path(), "scaleway-cli").is_ok());
}
#[test]
fn pending_and_last_version_are_tracked_independently() {
let dir = tempfile::tempdir().unwrap();
save_last_version(dir.path(), "scaleway-cli", "v2.61.0").unwrap();
save_pending_version(dir.path(), "scaleway-cli", "v2.62.0").unwrap();
assert_eq!(
load_last_version(dir.path(), "scaleway-cli"),
Some("v2.61.0".to_string())
);
assert_eq!(
load_pending_version(dir.path(), "scaleway-cli"),
Some("v2.62.0".to_string())
);
}
} }

21
src/test_support.rs Normal file
View file

@ -0,0 +1,21 @@
//! Test-only fixture helpers shared across modules' `#[cfg(test)]` code
//! (`publisher`, `sanity`) — not production code, and not built outside
//! `cargo test`. See docs/ARCHITECTURE.md > "organize by pipeline stage, not
//! by layer": this exists to remove one specific piece of duplication
//! (two near-identical copies of "write an executable shell script"), not
//! as a general test-utils dump.
use std::path::{Path, PathBuf};
/// Writes an executable `#!/bin/sh` script named `name` into `dir`,
/// running `body` as its contents. Used to stand in for a real binary
/// (`repo-add`, a package's own `--version` command) in tests, without
/// needing the real tool installed or a mutated global `PATH`.
pub(crate) fn write_executable_script(dir: &Path, name: &str, body: &str) -> PathBuf {
let path = dir.join(name);
std::fs::write(&path, format!("#!/bin/sh\n{body}\n")).unwrap();
let mut perms = std::fs::metadata(&path).unwrap().permissions();
std::os::unix::fs::PermissionsExt::set_mode(&mut perms, 0o755);
std::fs::set_permissions(&path, perms).unwrap();
path
}

26
src/verifier.rs
View file

@ -1,9 +1,14 @@
//! Runs the trust-tier-specific check declared for a package against a
//! downloaded artifact, and reports a pass/fail plus the tier it implies.
//! The only module that knows what each `Verification::method` actually
//! proves — see docs/SPEC.md > Verification trust tiers.
use crate::checker::version_from_tag; use crate::checker::version_from_tag;
use crate::config::Verification; use crate::config::Verification;
use crate::fetcher; use crate::fetcher;
use crate::github::GithubEndpoints; use crate::github::GithubEndpoints;
use crate::hash;
use anyhow::{Context, Result, bail}; use anyhow::{Context, Result, bail};
use sha2::{Digest, Sha256};
use std::path::Path; use std::path::Path;
use std::process::Command; use std::process::Command;
@ -14,7 +19,7 @@ pub struct VerificationResult {
} }
/// Runs the verification method declared for a package against a /// Runs the verification method declared for a package against a
/// downloaded artifact. See SPEC.md > Verification trust tiers for what /// downloaded artifact. See docs/SPEC.md > Verification trust tiers for what
/// each tier does and does not prove. /// each tier does and does not prove.
pub fn verify( pub fn verify(
client: &reqwest::blocking::Client, client: &reqwest::blocking::Client,
@ -31,7 +36,7 @@ pub fn verify(
} => { } => {
let checksum_asset_name = let checksum_asset_name =
checksum_asset_pattern.replace("{version}", version_from_tag(tag)); checksum_asset_pattern.replace("{version}", version_from_tag(tag));
let checksum_path = fetcher::download_asset( let checksum_asset = fetcher::download_asset(
client, client,
endpoints, endpoints,
repo, repo,
@ -39,15 +44,14 @@ pub fn verify(
&checksum_asset_name, &checksum_asset_name,
dest_dir, dest_dir,
)?; )?;
let checksum_text = std::fs::read_to_string(&checksum_path)?; let checksum_text = std::fs::read_to_string(&checksum_asset.path)?;
let artifact_name = artifact_path let artifact_name = artifact_path
.file_name() .file_name()
.and_then(|n| n.to_str()) .and_then(|n| n.to_str())
.context("artifact path has no filename")?; .context("artifact path has no filename")?;
let expected = expected_checksum(&checksum_text, artifact_name)?; let expected = expected_checksum(&checksum_text, artifact_name)?;
let data = std::fs::read(artifact_path)?; let actual = hash::sha256_hex_file(artifact_path)?;
let actual = sha256_hex(&data);
let passed = actual == expected; let passed = actual == expected;
Ok(VerificationResult { Ok(VerificationResult {
@ -55,7 +59,7 @@ pub fn verify(
passed, passed,
justification: if passed { justification: if passed {
"same-origin sha256 matched — proves transport integrity only, \ "same-origin sha256 matched — proves transport integrity only, \
not authorship (see tier 4 in SPEC.md)" not authorship (see tier 4 in docs/SPEC.md)"
.into() .into()
} else { } else {
format!("sha256 mismatch: expected {expected}, got {actual}") format!("sha256 mismatch: expected {expected}, got {actual}")
@ -94,12 +98,6 @@ pub fn verify(
} }
} }
fn sha256_hex(data: &[u8]) -> String {
let mut hasher = Sha256::new();
hasher.update(data);
hex::encode(hasher.finalize())
}
/// Finds the expected hash for `artifact_name` in a checksum file. /// Finds the expected hash for `artifact_name` in a checksum file.
/// ///
/// Handles both a bare-hash file covering a single asset (e.g. uv's /// Handles both a bare-hash file covering a single asset (e.g. uv's
@ -193,7 +191,7 @@ mod tests {
let dest_dir = tempfile::tempdir().unwrap(); let dest_dir = tempfile::tempdir().unwrap();
let artifact_path = dest_dir.path().join("thing.tar.gz"); let artifact_path = dest_dir.path().join("thing.tar.gz");
std::fs::write(&artifact_path, b"hello world").unwrap(); std::fs::write(&artifact_path, b"hello world").unwrap();
let expected_hash = sha256_hex(b"hello world"); let expected_hash = hash::sha256_hex(b"hello world");
let release_url = format!("{}/download/SHA256SUMS", server.url()); let release_url = format!("{}/download/SHA256SUMS", server.url());
let release_body = format!( let release_body = format!(