pkgwatch/src/publisher.rs

201 lines
8.2 KiB
Rust
Raw Normal View History

Add ARCHITECTURE.md and apply it to this PR's code Researched current industry practice on code organization/maintainability (Ousterhout's deep modules and information hiding, package-by-feature vs. package-by-layer, functional-core/imperative-shell testability, tech-debt prevention via ADR-equivalent inline rationale) and wrote it into ARCHITECTURE.md as a set of concrete, project-specific rules rather than a generic essay — each principle cites a real example already in this codebase or fixed by this commit. Cross-linked from SPEC.md, which stays about product design, not code organization. Applied it to this PR's own code: - Pulled process_package/fetch_and_verify/build_and_publish/run_review/ approve out of main.rs into a new pipeline.rs. main.rs's own main() had grown to 278 lines and zero tests by treating "it's just the entry point" as an excuse to skip separating logic from wiring; now main.rs is argv dispatch only. - Extracted decide_tier_action as a pure function (verification outcome + pending-state -> what to do), replacing dispatch logic that was previously inlined into a function that also made the real network/ build calls. Four unit tests, no I/O, covering all four outcomes. - Added a `//!` module doc comment to every file touched in this branch, each stating that module's one job in a sentence, per the "deep modules" principle the spec argues for. Coverage's reported total drops (94% -> 78%) because pipeline.rs is deliberately NOT excluded from it the way main.rs is, even though it's mostly the same kind of untestable I/O orchestration — excluding it would hide decide_tier_action's real unit-test coverage along with the untested parts. Noted inline in Makefile.toml/ci.yml so the number doesn't look like a quality regression at a glance. Also added a project reference memory pointing at ARCHITECTURE.md rather than duplicating its content there, per this session's own memory-hygiene rules (architecture/conventions are derivable from the repo and shouldn't be duplicated somewhere that can go stale). Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
2026-09-17 09:42:18 +00:00
//! Gets a built package into the local pacman repo: copies it in, runs
//! `repo-add`, and checks the repo is actually registered in
//! `/etc/pacman.conf` first. The only module that touches the repo
//! directory or pacman.conf.
Close the loop: build, sanity-check, and publish for the first time Implements the last unimplemented pipeline stage from SPEC.md: PKGBUILD generation + makepkg (builder.rs), a post-build version sanity check (sanity.rs), and repo-add publishing (publisher.rs), wired into main.rs for both the tier 1-3 auto-publish path and a new tier 4-6 review queue (`pkgwatch review` / `pkgwatch review <name> --approve`, persisted via state::{load,save,clear}_pending_version, tracked separately from last-published-version since approving one release isn't a standing auto-publish grant for future ones). Publishing targets an existing, already-registered local pacman repo (~/.local/share/pacman/custom, `[custom]` in /etc/pacman.conf) rather than one pkgwatch invents — found already in real use for a hand-packaged AppImage, which resolves SPEC's open question on where the repo lives without pkgwatch ever touching pacman.conf. Publishing stops at `repo-add`; actually installing/upgrading (`pacman -Syu`/`pacman -S`) is left to the operator, not run automatically. Getting a real second package (scaleway-cli, tier 4) through the new pipeline immediately surfaced a real gap: its pacman package is named `scaleway-cli` but the actual binary is `scw` (confirmed via `pacman -Ql` against the currently-installed extra package) — without a way to declare that, the build would install alongside extra's package under the wrong name instead of shadowing it. Added `Package::binary_name` (config.rs) to cover it. Every upstream-controlled string (version, asset name, download URL) is validated before it touches generated shell content in the PKGBUILD template — rejects anything containing a single quote or newline, since values are embedded in single-quoted bash strings. Verified for real, end to end: uv (tier 2) auto-built and published against the real astral-sh/uv release with no human step; scaleway-cli (tier 4) queued for review, then approved via `pkgwatch review scaleway-cli --approve`, which re-verified, built, and published it — confirmed the built package contains exactly usr/bin/scw. Both landed in the real custom repo's database. Left scaleway-cli's real-repo review pending rather than approving it myself: the tier 4-6 gate exists for a human judgment call, not the agent's. 69 tests, cargo make ci clean (fmt, clippy, complexity, coverage, audit). Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
2026-09-17 09:13:13 +00:00
use anyhow::{Context, Result, bail};
use std::path::{Path, PathBuf};
use std::process::Command;
/// Pacman's system-wide config — hardcoded like the rest of this tool's
/// Arch/Manjaro-specific assumptions (see SPEC.md > Scope).
const PACMAN_CONF: &str = "/etc/pacman.conf";
Close the loop: build, sanity-check, and publish for the first time Implements the last unimplemented pipeline stage from SPEC.md: PKGBUILD generation + makepkg (builder.rs), a post-build version sanity check (sanity.rs), and repo-add publishing (publisher.rs), wired into main.rs for both the tier 1-3 auto-publish path and a new tier 4-6 review queue (`pkgwatch review` / `pkgwatch review <name> --approve`, persisted via state::{load,save,clear}_pending_version, tracked separately from last-published-version since approving one release isn't a standing auto-publish grant for future ones). Publishing targets an existing, already-registered local pacman repo (~/.local/share/pacman/custom, `[custom]` in /etc/pacman.conf) rather than one pkgwatch invents — found already in real use for a hand-packaged AppImage, which resolves SPEC's open question on where the repo lives without pkgwatch ever touching pacman.conf. Publishing stops at `repo-add`; actually installing/upgrading (`pacman -Syu`/`pacman -S`) is left to the operator, not run automatically. Getting a real second package (scaleway-cli, tier 4) through the new pipeline immediately surfaced a real gap: its pacman package is named `scaleway-cli` but the actual binary is `scw` (confirmed via `pacman -Ql` against the currently-installed extra package) — without a way to declare that, the build would install alongside extra's package under the wrong name instead of shadowing it. Added `Package::binary_name` (config.rs) to cover it. Every upstream-controlled string (version, asset name, download URL) is validated before it touches generated shell content in the PKGBUILD template — rejects anything containing a single quote or newline, since values are embedded in single-quoted bash strings. Verified for real, end to end: uv (tier 2) auto-built and published against the real astral-sh/uv release with no human step; scaleway-cli (tier 4) queued for review, then approved via `pkgwatch review scaleway-cli --approve`, which re-verified, built, and published it — confirmed the built package contains exactly usr/bin/scw. Both landed in the real custom repo's database. Left scaleway-cli's real-repo review pending rather than approving it myself: the tier 4-6 gate exists for a human judgment call, not the agent's. 69 tests, cargo make ci clean (fmt, clippy, complexity, coverage, audit). Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
2026-09-17 09:13:13 +00:00
/// Copies the built package into `repo_dir` and runs `repo-add` against
/// `<repo_name>.db.tar.gz` there. Creates `repo_dir` if it doesn't exist
/// yet — `repo-add` itself creates the database file on its first run, so
/// everything filesystem-side is self-healing.
Close the loop: build, sanity-check, and publish for the first time Implements the last unimplemented pipeline stage from SPEC.md: PKGBUILD generation + makepkg (builder.rs), a post-build version sanity check (sanity.rs), and repo-add publishing (publisher.rs), wired into main.rs for both the tier 1-3 auto-publish path and a new tier 4-6 review queue (`pkgwatch review` / `pkgwatch review <name> --approve`, persisted via state::{load,save,clear}_pending_version, tracked separately from last-published-version since approving one release isn't a standing auto-publish grant for future ones). Publishing targets an existing, already-registered local pacman repo (~/.local/share/pacman/custom, `[custom]` in /etc/pacman.conf) rather than one pkgwatch invents — found already in real use for a hand-packaged AppImage, which resolves SPEC's open question on where the repo lives without pkgwatch ever touching pacman.conf. Publishing stops at `repo-add`; actually installing/upgrading (`pacman -Syu`/`pacman -S`) is left to the operator, not run automatically. Getting a real second package (scaleway-cli, tier 4) through the new pipeline immediately surfaced a real gap: its pacman package is named `scaleway-cli` but the actual binary is `scw` (confirmed via `pacman -Ql` against the currently-installed extra package) — without a way to declare that, the build would install alongside extra's package under the wrong name instead of shadowing it. Added `Package::binary_name` (config.rs) to cover it. Every upstream-controlled string (version, asset name, download URL) is validated before it touches generated shell content in the PKGBUILD template — rejects anything containing a single quote or newline, since values are embedded in single-quoted bash strings. Verified for real, end to end: uv (tier 2) auto-built and published against the real astral-sh/uv release with no human step; scaleway-cli (tier 4) queued for review, then approved via `pkgwatch review scaleway-cli --approve`, which re-verified, built, and published it — confirmed the built package contains exactly usr/bin/scw. Both landed in the real custom repo's database. Left scaleway-cli's real-repo review pending rather than approving it myself: the tier 4-6 gate exists for a human judgment call, not the agent's. 69 tests, cargo make ci clean (fmt, clippy, complexity, coverage, audit). Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
2026-09-17 09:13:13 +00:00
///
/// What is *not* self-healing, and can't safely be: registering
/// `repo_name` in `/etc/pacman.conf` (see `ensure_registered`) — that
/// needs root, which this process doesn't have and shouldn't grab for
/// itself. Getting a published version onto the running system is a
/// separate, deliberate `pacman -Syu`/`pacman -S` step too, also left to
/// the operator.
Close the loop: build, sanity-check, and publish for the first time Implements the last unimplemented pipeline stage from SPEC.md: PKGBUILD generation + makepkg (builder.rs), a post-build version sanity check (sanity.rs), and repo-add publishing (publisher.rs), wired into main.rs for both the tier 1-3 auto-publish path and a new tier 4-6 review queue (`pkgwatch review` / `pkgwatch review <name> --approve`, persisted via state::{load,save,clear}_pending_version, tracked separately from last-published-version since approving one release isn't a standing auto-publish grant for future ones). Publishing targets an existing, already-registered local pacman repo (~/.local/share/pacman/custom, `[custom]` in /etc/pacman.conf) rather than one pkgwatch invents — found already in real use for a hand-packaged AppImage, which resolves SPEC's open question on where the repo lives without pkgwatch ever touching pacman.conf. Publishing stops at `repo-add`; actually installing/upgrading (`pacman -Syu`/`pacman -S`) is left to the operator, not run automatically. Getting a real second package (scaleway-cli, tier 4) through the new pipeline immediately surfaced a real gap: its pacman package is named `scaleway-cli` but the actual binary is `scw` (confirmed via `pacman -Ql` against the currently-installed extra package) — without a way to declare that, the build would install alongside extra's package under the wrong name instead of shadowing it. Added `Package::binary_name` (config.rs) to cover it. Every upstream-controlled string (version, asset name, download URL) is validated before it touches generated shell content in the PKGBUILD template — rejects anything containing a single quote or newline, since values are embedded in single-quoted bash strings. Verified for real, end to end: uv (tier 2) auto-built and published against the real astral-sh/uv release with no human step; scaleway-cli (tier 4) queued for review, then approved via `pkgwatch review scaleway-cli --approve`, which re-verified, built, and published it — confirmed the built package contains exactly usr/bin/scw. Both landed in the real custom repo's database. Left scaleway-cli's real-repo review pending rather than approving it myself: the tier 4-6 gate exists for a human judgment call, not the agent's. 69 tests, cargo make ci clean (fmt, clippy, complexity, coverage, audit). Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
2026-09-17 09:13:13 +00:00
pub fn publish(package_path: &Path, repo_dir: &Path, repo_name: &str) -> Result<PathBuf> {
publish_with(Path::new("repo-add"), package_path, repo_dir, repo_name)
}
/// `repo_add_bin` is injectable so tests can point it at a stub script
/// instead of the real `repo-add` (or a mutated global `PATH`, which would
/// race with `cargo test`'s parallel test threads).
fn publish_with(
repo_add_bin: &Path,
package_path: &Path,
repo_dir: &Path,
repo_name: &str,
) -> Result<PathBuf> {
std::fs::create_dir_all(repo_dir)
.with_context(|| format!("creating repo dir {}", repo_dir.display()))?;
let file_name = package_path
.file_name()
.context("built package path has no filename")?;
let dest = repo_dir.join(file_name);
std::fs::copy(package_path, &dest)
.with_context(|| format!("copying {} to {}", package_path.display(), dest.display()))?;
let db_path = repo_dir.join(format!("{repo_name}.db.tar.gz"));
let status = Command::new(repo_add_bin)
.arg(&db_path)
.arg(&dest)
.status()
.with_context(|| {
format!(
"running {} (is pacman-contrib installed?)",
repo_add_bin.display()
)
})?;
if !status.success() {
bail!("repo-add failed for {}", dest.display());
}
Ok(dest)
}
/// Verifies `repo_name` is registered as an active `[section]` in
/// `/etc/pacman.conf`, so a build isn't wasted on a repo pacman will never
/// actually sync from. Call this before `publish` — ideally before even
/// starting the build, so a missing repo fails fast instead of after
/// several seconds of `makepkg` work.
///
/// Doesn't check that the section's `Server =`/`Include =` line points at
/// `repo_dir` specifically — just that a repo by this name exists at all.
/// A same-named repo pointed somewhere else is a rare, easily-diagnosed
/// misconfiguration, not worth the parsing complexity to catch here.
pub fn ensure_registered(repo_name: &str, repo_dir: &Path) -> Result<()> {
ensure_registered_at(Path::new(PACMAN_CONF), repo_name, repo_dir)
}
fn ensure_registered_at(pacman_conf: &Path, repo_name: &str, repo_dir: &Path) -> Result<()> {
let conf = std::fs::read_to_string(pacman_conf)
.with_context(|| format!("reading {}", pacman_conf.display()))?;
let header = format!("[{repo_name}]");
let registered = conf.lines().map(str::trim).any(|line| line == header);
if !registered {
bail!(
"'{repo_name}' is not registered in {} — add this once, as root, then re-run:\n\n\
[{repo_name}]\n\
SigLevel = Optional TrustAll\n\
Server = file://{}\n",
pacman_conf.display(),
repo_dir.display()
);
}
Ok(())
}
Close the loop: build, sanity-check, and publish for the first time Implements the last unimplemented pipeline stage from SPEC.md: PKGBUILD generation + makepkg (builder.rs), a post-build version sanity check (sanity.rs), and repo-add publishing (publisher.rs), wired into main.rs for both the tier 1-3 auto-publish path and a new tier 4-6 review queue (`pkgwatch review` / `pkgwatch review <name> --approve`, persisted via state::{load,save,clear}_pending_version, tracked separately from last-published-version since approving one release isn't a standing auto-publish grant for future ones). Publishing targets an existing, already-registered local pacman repo (~/.local/share/pacman/custom, `[custom]` in /etc/pacman.conf) rather than one pkgwatch invents — found already in real use for a hand-packaged AppImage, which resolves SPEC's open question on where the repo lives without pkgwatch ever touching pacman.conf. Publishing stops at `repo-add`; actually installing/upgrading (`pacman -Syu`/`pacman -S`) is left to the operator, not run automatically. Getting a real second package (scaleway-cli, tier 4) through the new pipeline immediately surfaced a real gap: its pacman package is named `scaleway-cli` but the actual binary is `scw` (confirmed via `pacman -Ql` against the currently-installed extra package) — without a way to declare that, the build would install alongside extra's package under the wrong name instead of shadowing it. Added `Package::binary_name` (config.rs) to cover it. Every upstream-controlled string (version, asset name, download URL) is validated before it touches generated shell content in the PKGBUILD template — rejects anything containing a single quote or newline, since values are embedded in single-quoted bash strings. Verified for real, end to end: uv (tier 2) auto-built and published against the real astral-sh/uv release with no human step; scaleway-cli (tier 4) queued for review, then approved via `pkgwatch review scaleway-cli --approve`, which re-verified, built, and published it — confirmed the built package contains exactly usr/bin/scw. Both landed in the real custom repo's database. Left scaleway-cli's real-repo review pending rather than approving it myself: the tier 4-6 gate exists for a human judgment call, not the agent's. 69 tests, cargo make ci clean (fmt, clippy, complexity, coverage, audit). Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
2026-09-17 09:13:13 +00:00
#[cfg(test)]
mod tests {
use super::*;
Fix issues from code review: shell-escaping gap, exit code, and more A single-agent code review of this branch's diff (builder/pipeline/ publisher/sanity/hash + main/config/fetcher/state/verifier changes) found six real issues, all fixed here: - builder.rs: validate_shell_safe only rejected a literal single quote and newline, written for the single-quoted PKGBUILD fields. But asset_name (via install_source) and binary_name land in the install() line, which is necessarily double-quoted so ${srcdir}/${pkgdir} can expand — where $, backtick, and backslash are still live. Not currently exploitable (the one variable component, version, is already independently constrained by validate_pkgver's strict charset), but a latent gap relying on that coincidence rather than the validator actually covering its real use context. Widened the reject-list to cover both quoting styles, added regression tests including one at the generate_pkgbuild level. Corrected SPEC.md's "single-quoted" claim to match. - pipeline.rs: a verification failure returned Ok(()) from process_package, so run_check never counted it as a failure and the process exited 0 even on a failed cryptographic/attestation check — exactly the event a monitoring setup (systemd OnFailure=, cron mail-on-error) most needs a non-zero exit to catch. Now bails, which run_check already treats as a package failure. Added an integration test against a mocked GitHub server exercising this exact path. - builder.rs: find_built_package hardcoded the .pkg.tar.zst suffix, so a box with a different PKGEXT in makepkg.conf would report a false "makepkg failed" for a build that actually succeeded. Widened to match any .pkg.tar.* compression. Added direct unit tests (it had none). - pipeline.rs: a newer tier 4-6 version silently overwrote a still- unreviewed older pending version with no indication anything was superseded. Now says so explicitly. - hash.rs: builder/verifier each read a whole downloaded artifact into memory via std::fs::read just to hash it, doubling peak memory for no reason since the file's already on disk. Added sha256_hex_file, streamed in fixed-size chunks; both callers switched to it. - Deduplicated two near-identical test-only "write an executable shell script" helpers (publisher.rs, sanity.rs) into a shared src/test_support.rs. 75 tests (was 63), cargo make ci clean. Re-verified end to end against the real astral-sh/uv release after all six fixes — build, sanity check, and publish into a scratch repo all still succeed. Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
2026-09-17 10:06:40 +00:00
use crate::test_support::write_executable_script;
Close the loop: build, sanity-check, and publish for the first time Implements the last unimplemented pipeline stage from SPEC.md: PKGBUILD generation + makepkg (builder.rs), a post-build version sanity check (sanity.rs), and repo-add publishing (publisher.rs), wired into main.rs for both the tier 1-3 auto-publish path and a new tier 4-6 review queue (`pkgwatch review` / `pkgwatch review <name> --approve`, persisted via state::{load,save,clear}_pending_version, tracked separately from last-published-version since approving one release isn't a standing auto-publish grant for future ones). Publishing targets an existing, already-registered local pacman repo (~/.local/share/pacman/custom, `[custom]` in /etc/pacman.conf) rather than one pkgwatch invents — found already in real use for a hand-packaged AppImage, which resolves SPEC's open question on where the repo lives without pkgwatch ever touching pacman.conf. Publishing stops at `repo-add`; actually installing/upgrading (`pacman -Syu`/`pacman -S`) is left to the operator, not run automatically. Getting a real second package (scaleway-cli, tier 4) through the new pipeline immediately surfaced a real gap: its pacman package is named `scaleway-cli` but the actual binary is `scw` (confirmed via `pacman -Ql` against the currently-installed extra package) — without a way to declare that, the build would install alongside extra's package under the wrong name instead of shadowing it. Added `Package::binary_name` (config.rs) to cover it. Every upstream-controlled string (version, asset name, download URL) is validated before it touches generated shell content in the PKGBUILD template — rejects anything containing a single quote or newline, since values are embedded in single-quoted bash strings. Verified for real, end to end: uv (tier 2) auto-built and published against the real astral-sh/uv release with no human step; scaleway-cli (tier 4) queued for review, then approved via `pkgwatch review scaleway-cli --approve`, which re-verified, built, and published it — confirmed the built package contains exactly usr/bin/scw. Both landed in the real custom repo's database. Left scaleway-cli's real-repo review pending rather than approving it myself: the tier 4-6 gate exists for a human judgment call, not the agent's. 69 tests, cargo make ci clean (fmt, clippy, complexity, coverage, audit). Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
2026-09-17 09:13:13 +00:00
#[test]
fn publish_copies_package_and_invokes_repo_add() {
let stub_dir = tempfile::tempdir().unwrap();
let log_path = stub_dir.path().join("invoked_with.txt");
Fix issues from code review: shell-escaping gap, exit code, and more A single-agent code review of this branch's diff (builder/pipeline/ publisher/sanity/hash + main/config/fetcher/state/verifier changes) found six real issues, all fixed here: - builder.rs: validate_shell_safe only rejected a literal single quote and newline, written for the single-quoted PKGBUILD fields. But asset_name (via install_source) and binary_name land in the install() line, which is necessarily double-quoted so ${srcdir}/${pkgdir} can expand — where $, backtick, and backslash are still live. Not currently exploitable (the one variable component, version, is already independently constrained by validate_pkgver's strict charset), but a latent gap relying on that coincidence rather than the validator actually covering its real use context. Widened the reject-list to cover both quoting styles, added regression tests including one at the generate_pkgbuild level. Corrected SPEC.md's "single-quoted" claim to match. - pipeline.rs: a verification failure returned Ok(()) from process_package, so run_check never counted it as a failure and the process exited 0 even on a failed cryptographic/attestation check — exactly the event a monitoring setup (systemd OnFailure=, cron mail-on-error) most needs a non-zero exit to catch. Now bails, which run_check already treats as a package failure. Added an integration test against a mocked GitHub server exercising this exact path. - builder.rs: find_built_package hardcoded the .pkg.tar.zst suffix, so a box with a different PKGEXT in makepkg.conf would report a false "makepkg failed" for a build that actually succeeded. Widened to match any .pkg.tar.* compression. Added direct unit tests (it had none). - pipeline.rs: a newer tier 4-6 version silently overwrote a still- unreviewed older pending version with no indication anything was superseded. Now says so explicitly. - hash.rs: builder/verifier each read a whole downloaded artifact into memory via std::fs::read just to hash it, doubling peak memory for no reason since the file's already on disk. Added sha256_hex_file, streamed in fixed-size chunks; both callers switched to it. - Deduplicated two near-identical test-only "write an executable shell script" helpers (publisher.rs, sanity.rs) into a shared src/test_support.rs. 75 tests (was 63), cargo make ci clean. Re-verified end to end against the real astral-sh/uv release after all six fixes — build, sanity check, and publish into a scratch repo all still succeed. Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
2026-09-17 10:06:40 +00:00
let repo_add = write_executable_script(
Close the loop: build, sanity-check, and publish for the first time Implements the last unimplemented pipeline stage from SPEC.md: PKGBUILD generation + makepkg (builder.rs), a post-build version sanity check (sanity.rs), and repo-add publishing (publisher.rs), wired into main.rs for both the tier 1-3 auto-publish path and a new tier 4-6 review queue (`pkgwatch review` / `pkgwatch review <name> --approve`, persisted via state::{load,save,clear}_pending_version, tracked separately from last-published-version since approving one release isn't a standing auto-publish grant for future ones). Publishing targets an existing, already-registered local pacman repo (~/.local/share/pacman/custom, `[custom]` in /etc/pacman.conf) rather than one pkgwatch invents — found already in real use for a hand-packaged AppImage, which resolves SPEC's open question on where the repo lives without pkgwatch ever touching pacman.conf. Publishing stops at `repo-add`; actually installing/upgrading (`pacman -Syu`/`pacman -S`) is left to the operator, not run automatically. Getting a real second package (scaleway-cli, tier 4) through the new pipeline immediately surfaced a real gap: its pacman package is named `scaleway-cli` but the actual binary is `scw` (confirmed via `pacman -Ql` against the currently-installed extra package) — without a way to declare that, the build would install alongside extra's package under the wrong name instead of shadowing it. Added `Package::binary_name` (config.rs) to cover it. Every upstream-controlled string (version, asset name, download URL) is validated before it touches generated shell content in the PKGBUILD template — rejects anything containing a single quote or newline, since values are embedded in single-quoted bash strings. Verified for real, end to end: uv (tier 2) auto-built and published against the real astral-sh/uv release with no human step; scaleway-cli (tier 4) queued for review, then approved via `pkgwatch review scaleway-cli --approve`, which re-verified, built, and published it — confirmed the built package contains exactly usr/bin/scw. Both landed in the real custom repo's database. Left scaleway-cli's real-repo review pending rather than approving it myself: the tier 4-6 gate exists for a human judgment call, not the agent's. 69 tests, cargo make ci clean (fmt, clippy, complexity, coverage, audit). Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
2026-09-17 09:13:13 +00:00
stub_dir.path(),
"fake-repo-add",
&format!("echo \"$@\" > {}", log_path.display()),
);
let src_dir = tempfile::tempdir().unwrap();
let package_path = src_dir.path().join("thing-1.0.0-1-x86_64.pkg.tar.zst");
std::fs::write(&package_path, b"pkg-bytes").unwrap();
let repo_dir = tempfile::tempdir().unwrap();
let dest = publish_with(&repo_add, &package_path, repo_dir.path(), "custom").unwrap();
assert_eq!(
dest,
repo_dir.path().join("thing-1.0.0-1-x86_64.pkg.tar.zst")
);
assert_eq!(std::fs::read(&dest).unwrap(), b"pkg-bytes");
let invoked_with = std::fs::read_to_string(&log_path).unwrap();
assert!(invoked_with.contains("custom.db.tar.gz"));
assert!(invoked_with.contains("thing-1.0.0-1-x86_64.pkg.tar.zst"));
}
#[test]
fn publish_errors_when_repo_add_fails() {
let stub_dir = tempfile::tempdir().unwrap();
Fix issues from code review: shell-escaping gap, exit code, and more A single-agent code review of this branch's diff (builder/pipeline/ publisher/sanity/hash + main/config/fetcher/state/verifier changes) found six real issues, all fixed here: - builder.rs: validate_shell_safe only rejected a literal single quote and newline, written for the single-quoted PKGBUILD fields. But asset_name (via install_source) and binary_name land in the install() line, which is necessarily double-quoted so ${srcdir}/${pkgdir} can expand — where $, backtick, and backslash are still live. Not currently exploitable (the one variable component, version, is already independently constrained by validate_pkgver's strict charset), but a latent gap relying on that coincidence rather than the validator actually covering its real use context. Widened the reject-list to cover both quoting styles, added regression tests including one at the generate_pkgbuild level. Corrected SPEC.md's "single-quoted" claim to match. - pipeline.rs: a verification failure returned Ok(()) from process_package, so run_check never counted it as a failure and the process exited 0 even on a failed cryptographic/attestation check — exactly the event a monitoring setup (systemd OnFailure=, cron mail-on-error) most needs a non-zero exit to catch. Now bails, which run_check already treats as a package failure. Added an integration test against a mocked GitHub server exercising this exact path. - builder.rs: find_built_package hardcoded the .pkg.tar.zst suffix, so a box with a different PKGEXT in makepkg.conf would report a false "makepkg failed" for a build that actually succeeded. Widened to match any .pkg.tar.* compression. Added direct unit tests (it had none). - pipeline.rs: a newer tier 4-6 version silently overwrote a still- unreviewed older pending version with no indication anything was superseded. Now says so explicitly. - hash.rs: builder/verifier each read a whole downloaded artifact into memory via std::fs::read just to hash it, doubling peak memory for no reason since the file's already on disk. Added sha256_hex_file, streamed in fixed-size chunks; both callers switched to it. - Deduplicated two near-identical test-only "write an executable shell script" helpers (publisher.rs, sanity.rs) into a shared src/test_support.rs. 75 tests (was 63), cargo make ci clean. Re-verified end to end against the real astral-sh/uv release after all six fixes — build, sanity check, and publish into a scratch repo all still succeed. Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
2026-09-17 10:06:40 +00:00
let repo_add = write_executable_script(stub_dir.path(), "fake-repo-add-fail", "exit 1");
Close the loop: build, sanity-check, and publish for the first time Implements the last unimplemented pipeline stage from SPEC.md: PKGBUILD generation + makepkg (builder.rs), a post-build version sanity check (sanity.rs), and repo-add publishing (publisher.rs), wired into main.rs for both the tier 1-3 auto-publish path and a new tier 4-6 review queue (`pkgwatch review` / `pkgwatch review <name> --approve`, persisted via state::{load,save,clear}_pending_version, tracked separately from last-published-version since approving one release isn't a standing auto-publish grant for future ones). Publishing targets an existing, already-registered local pacman repo (~/.local/share/pacman/custom, `[custom]` in /etc/pacman.conf) rather than one pkgwatch invents — found already in real use for a hand-packaged AppImage, which resolves SPEC's open question on where the repo lives without pkgwatch ever touching pacman.conf. Publishing stops at `repo-add`; actually installing/upgrading (`pacman -Syu`/`pacman -S`) is left to the operator, not run automatically. Getting a real second package (scaleway-cli, tier 4) through the new pipeline immediately surfaced a real gap: its pacman package is named `scaleway-cli` but the actual binary is `scw` (confirmed via `pacman -Ql` against the currently-installed extra package) — without a way to declare that, the build would install alongside extra's package under the wrong name instead of shadowing it. Added `Package::binary_name` (config.rs) to cover it. Every upstream-controlled string (version, asset name, download URL) is validated before it touches generated shell content in the PKGBUILD template — rejects anything containing a single quote or newline, since values are embedded in single-quoted bash strings. Verified for real, end to end: uv (tier 2) auto-built and published against the real astral-sh/uv release with no human step; scaleway-cli (tier 4) queued for review, then approved via `pkgwatch review scaleway-cli --approve`, which re-verified, built, and published it — confirmed the built package contains exactly usr/bin/scw. Both landed in the real custom repo's database. Left scaleway-cli's real-repo review pending rather than approving it myself: the tier 4-6 gate exists for a human judgment call, not the agent's. 69 tests, cargo make ci clean (fmt, clippy, complexity, coverage, audit). Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
2026-09-17 09:13:13 +00:00
let src_dir = tempfile::tempdir().unwrap();
let package_path = src_dir.path().join("thing-1.0.0-1-x86_64.pkg.tar.zst");
std::fs::write(&package_path, b"pkg-bytes").unwrap();
let repo_dir = tempfile::tempdir().unwrap();
let err = publish_with(&repo_add, &package_path, repo_dir.path(), "custom").unwrap_err();
assert!(err.to_string().contains("repo-add failed"));
}
#[test]
fn publish_creates_repo_dir_if_missing() {
let stub_dir = tempfile::tempdir().unwrap();
Fix issues from code review: shell-escaping gap, exit code, and more A single-agent code review of this branch's diff (builder/pipeline/ publisher/sanity/hash + main/config/fetcher/state/verifier changes) found six real issues, all fixed here: - builder.rs: validate_shell_safe only rejected a literal single quote and newline, written for the single-quoted PKGBUILD fields. But asset_name (via install_source) and binary_name land in the install() line, which is necessarily double-quoted so ${srcdir}/${pkgdir} can expand — where $, backtick, and backslash are still live. Not currently exploitable (the one variable component, version, is already independently constrained by validate_pkgver's strict charset), but a latent gap relying on that coincidence rather than the validator actually covering its real use context. Widened the reject-list to cover both quoting styles, added regression tests including one at the generate_pkgbuild level. Corrected SPEC.md's "single-quoted" claim to match. - pipeline.rs: a verification failure returned Ok(()) from process_package, so run_check never counted it as a failure and the process exited 0 even on a failed cryptographic/attestation check — exactly the event a monitoring setup (systemd OnFailure=, cron mail-on-error) most needs a non-zero exit to catch. Now bails, which run_check already treats as a package failure. Added an integration test against a mocked GitHub server exercising this exact path. - builder.rs: find_built_package hardcoded the .pkg.tar.zst suffix, so a box with a different PKGEXT in makepkg.conf would report a false "makepkg failed" for a build that actually succeeded. Widened to match any .pkg.tar.* compression. Added direct unit tests (it had none). - pipeline.rs: a newer tier 4-6 version silently overwrote a still- unreviewed older pending version with no indication anything was superseded. Now says so explicitly. - hash.rs: builder/verifier each read a whole downloaded artifact into memory via std::fs::read just to hash it, doubling peak memory for no reason since the file's already on disk. Added sha256_hex_file, streamed in fixed-size chunks; both callers switched to it. - Deduplicated two near-identical test-only "write an executable shell script" helpers (publisher.rs, sanity.rs) into a shared src/test_support.rs. 75 tests (was 63), cargo make ci clean. Re-verified end to end against the real astral-sh/uv release after all six fixes — build, sanity check, and publish into a scratch repo all still succeed. Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
2026-09-17 10:06:40 +00:00
let repo_add = write_executable_script(stub_dir.path(), "fake-repo-add-ok", "exit 0");
Close the loop: build, sanity-check, and publish for the first time Implements the last unimplemented pipeline stage from SPEC.md: PKGBUILD generation + makepkg (builder.rs), a post-build version sanity check (sanity.rs), and repo-add publishing (publisher.rs), wired into main.rs for both the tier 1-3 auto-publish path and a new tier 4-6 review queue (`pkgwatch review` / `pkgwatch review <name> --approve`, persisted via state::{load,save,clear}_pending_version, tracked separately from last-published-version since approving one release isn't a standing auto-publish grant for future ones). Publishing targets an existing, already-registered local pacman repo (~/.local/share/pacman/custom, `[custom]` in /etc/pacman.conf) rather than one pkgwatch invents — found already in real use for a hand-packaged AppImage, which resolves SPEC's open question on where the repo lives without pkgwatch ever touching pacman.conf. Publishing stops at `repo-add`; actually installing/upgrading (`pacman -Syu`/`pacman -S`) is left to the operator, not run automatically. Getting a real second package (scaleway-cli, tier 4) through the new pipeline immediately surfaced a real gap: its pacman package is named `scaleway-cli` but the actual binary is `scw` (confirmed via `pacman -Ql` against the currently-installed extra package) — without a way to declare that, the build would install alongside extra's package under the wrong name instead of shadowing it. Added `Package::binary_name` (config.rs) to cover it. Every upstream-controlled string (version, asset name, download URL) is validated before it touches generated shell content in the PKGBUILD template — rejects anything containing a single quote or newline, since values are embedded in single-quoted bash strings. Verified for real, end to end: uv (tier 2) auto-built and published against the real astral-sh/uv release with no human step; scaleway-cli (tier 4) queued for review, then approved via `pkgwatch review scaleway-cli --approve`, which re-verified, built, and published it — confirmed the built package contains exactly usr/bin/scw. Both landed in the real custom repo's database. Left scaleway-cli's real-repo review pending rather than approving it myself: the tier 4-6 gate exists for a human judgment call, not the agent's. 69 tests, cargo make ci clean (fmt, clippy, complexity, coverage, audit). Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
2026-09-17 09:13:13 +00:00
let src_dir = tempfile::tempdir().unwrap();
let package_path = src_dir.path().join("thing-1.0.0-1-x86_64.pkg.tar.zst");
std::fs::write(&package_path, b"pkg-bytes").unwrap();
let parent = tempfile::tempdir().unwrap();
let repo_dir = parent.path().join("nested/repo");
publish_with(&repo_add, &package_path, &repo_dir, "custom").unwrap();
assert!(repo_dir.join("thing-1.0.0-1-x86_64.pkg.tar.zst").exists());
}
fn write_pacman_conf(dir: &Path, contents: &str) -> PathBuf {
let path = dir.join("pacman.conf");
std::fs::write(&path, contents).unwrap();
path
}
#[test]
fn ensure_registered_passes_when_section_present() {
let dir = tempfile::tempdir().unwrap();
let conf = write_pacman_conf(
dir.path(),
"[options]\nArchitecture = auto\n\n[extra]\nInclude = /etc/pacman.d/mirrorlist\n\n[custom]\nSigLevel = Optional TrustAll\nServer = file:///home/austin/.local/share/pacman/custom\n",
);
assert!(ensure_registered_at(&conf, "custom", Path::new("/repo")).is_ok());
}
#[test]
fn ensure_registered_fails_when_section_missing() {
let dir = tempfile::tempdir().unwrap();
let conf = write_pacman_conf(dir.path(), "[options]\nArchitecture = auto\n\n[extra]\n");
let err = ensure_registered_at(&conf, "custom", Path::new("/repo")).unwrap_err();
let msg = err.to_string();
assert!(msg.contains("not registered"));
assert!(msg.contains("[custom]"));
assert!(msg.contains("/repo"));
}
#[test]
fn ensure_registered_does_not_match_substring_of_another_section() {
// "custom" must match the whole section header, not just appear
// as a substring of e.g. "[custom-extra]".
let dir = tempfile::tempdir().unwrap();
let conf = write_pacman_conf(dir.path(), "[custom-extra]\nServer = file:///elsewhere\n");
assert!(ensure_registered_at(&conf, "custom", Path::new("/repo")).is_err());
}
#[test]
fn ensure_registered_errors_when_pacman_conf_missing() {
let dir = tempfile::tempdir().unwrap();
let missing = dir.path().join("does-not-exist.conf");
assert!(ensure_registered_at(&missing, "custom", Path::new("/repo")).is_err());
}
Close the loop: build, sanity-check, and publish for the first time Implements the last unimplemented pipeline stage from SPEC.md: PKGBUILD generation + makepkg (builder.rs), a post-build version sanity check (sanity.rs), and repo-add publishing (publisher.rs), wired into main.rs for both the tier 1-3 auto-publish path and a new tier 4-6 review queue (`pkgwatch review` / `pkgwatch review <name> --approve`, persisted via state::{load,save,clear}_pending_version, tracked separately from last-published-version since approving one release isn't a standing auto-publish grant for future ones). Publishing targets an existing, already-registered local pacman repo (~/.local/share/pacman/custom, `[custom]` in /etc/pacman.conf) rather than one pkgwatch invents — found already in real use for a hand-packaged AppImage, which resolves SPEC's open question on where the repo lives without pkgwatch ever touching pacman.conf. Publishing stops at `repo-add`; actually installing/upgrading (`pacman -Syu`/`pacman -S`) is left to the operator, not run automatically. Getting a real second package (scaleway-cli, tier 4) through the new pipeline immediately surfaced a real gap: its pacman package is named `scaleway-cli` but the actual binary is `scw` (confirmed via `pacman -Ql` against the currently-installed extra package) — without a way to declare that, the build would install alongside extra's package under the wrong name instead of shadowing it. Added `Package::binary_name` (config.rs) to cover it. Every upstream-controlled string (version, asset name, download URL) is validated before it touches generated shell content in the PKGBUILD template — rejects anything containing a single quote or newline, since values are embedded in single-quoted bash strings. Verified for real, end to end: uv (tier 2) auto-built and published against the real astral-sh/uv release with no human step; scaleway-cli (tier 4) queued for review, then approved via `pkgwatch review scaleway-cli --approve`, which re-verified, built, and published it — confirmed the built package contains exactly usr/bin/scw. Both landed in the real custom repo's database. Left scaleway-cli's real-repo review pending rather than approving it myself: the tier 4-6 gate exists for a human judgment call, not the agent's. 69 tests, cargo make ci clean (fmt, clippy, complexity, coverage, audit). Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
2026-09-17 09:13:13 +00:00
}