pkgwatch/src/publisher.rs

204 lines
8.2 KiB
Rust
Raw Normal View History

Close the loop: build, sanity-check, and publish for the first time Implements the last unimplemented pipeline stage from SPEC.md: PKGBUILD generation + makepkg (builder.rs), a post-build version sanity check (sanity.rs), and repo-add publishing (publisher.rs), wired into main.rs for both the tier 1-3 auto-publish path and a new tier 4-6 review queue (`pkgwatch review` / `pkgwatch review <name> --approve`, persisted via state::{load,save,clear}_pending_version, tracked separately from last-published-version since approving one release isn't a standing auto-publish grant for future ones). Publishing targets an existing, already-registered local pacman repo (~/.local/share/pacman/custom, `[custom]` in /etc/pacman.conf) rather than one pkgwatch invents — found already in real use for a hand-packaged AppImage, which resolves SPEC's open question on where the repo lives without pkgwatch ever touching pacman.conf. Publishing stops at `repo-add`; actually installing/upgrading (`pacman -Syu`/`pacman -S`) is left to the operator, not run automatically. Getting a real second package (scaleway-cli, tier 4) through the new pipeline immediately surfaced a real gap: its pacman package is named `scaleway-cli` but the actual binary is `scw` (confirmed via `pacman -Ql` against the currently-installed extra package) — without a way to declare that, the build would install alongside extra's package under the wrong name instead of shadowing it. Added `Package::binary_name` (config.rs) to cover it. Every upstream-controlled string (version, asset name, download URL) is validated before it touches generated shell content in the PKGBUILD template — rejects anything containing a single quote or newline, since values are embedded in single-quoted bash strings. Verified for real, end to end: uv (tier 2) auto-built and published against the real astral-sh/uv release with no human step; scaleway-cli (tier 4) queued for review, then approved via `pkgwatch review scaleway-cli --approve`, which re-verified, built, and published it — confirmed the built package contains exactly usr/bin/scw. Both landed in the real custom repo's database. Left scaleway-cli's real-repo review pending rather than approving it myself: the tier 4-6 gate exists for a human judgment call, not the agent's. 69 tests, cargo make ci clean (fmt, clippy, complexity, coverage, audit). Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
2026-09-17 09:13:13 +00:00
use anyhow::{Context, Result, bail};
use std::path::{Path, PathBuf};
use std::process::Command;
/// Pacman's system-wide config — hardcoded like the rest of this tool's
/// Arch/Manjaro-specific assumptions (see SPEC.md > Scope).
const PACMAN_CONF: &str = "/etc/pacman.conf";
Close the loop: build, sanity-check, and publish for the first time Implements the last unimplemented pipeline stage from SPEC.md: PKGBUILD generation + makepkg (builder.rs), a post-build version sanity check (sanity.rs), and repo-add publishing (publisher.rs), wired into main.rs for both the tier 1-3 auto-publish path and a new tier 4-6 review queue (`pkgwatch review` / `pkgwatch review <name> --approve`, persisted via state::{load,save,clear}_pending_version, tracked separately from last-published-version since approving one release isn't a standing auto-publish grant for future ones). Publishing targets an existing, already-registered local pacman repo (~/.local/share/pacman/custom, `[custom]` in /etc/pacman.conf) rather than one pkgwatch invents — found already in real use for a hand-packaged AppImage, which resolves SPEC's open question on where the repo lives without pkgwatch ever touching pacman.conf. Publishing stops at `repo-add`; actually installing/upgrading (`pacman -Syu`/`pacman -S`) is left to the operator, not run automatically. Getting a real second package (scaleway-cli, tier 4) through the new pipeline immediately surfaced a real gap: its pacman package is named `scaleway-cli` but the actual binary is `scw` (confirmed via `pacman -Ql` against the currently-installed extra package) — without a way to declare that, the build would install alongside extra's package under the wrong name instead of shadowing it. Added `Package::binary_name` (config.rs) to cover it. Every upstream-controlled string (version, asset name, download URL) is validated before it touches generated shell content in the PKGBUILD template — rejects anything containing a single quote or newline, since values are embedded in single-quoted bash strings. Verified for real, end to end: uv (tier 2) auto-built and published against the real astral-sh/uv release with no human step; scaleway-cli (tier 4) queued for review, then approved via `pkgwatch review scaleway-cli --approve`, which re-verified, built, and published it — confirmed the built package contains exactly usr/bin/scw. Both landed in the real custom repo's database. Left scaleway-cli's real-repo review pending rather than approving it myself: the tier 4-6 gate exists for a human judgment call, not the agent's. 69 tests, cargo make ci clean (fmt, clippy, complexity, coverage, audit). Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
2026-09-17 09:13:13 +00:00
/// Copies the built package into `repo_dir` and runs `repo-add` against
/// `<repo_name>.db.tar.gz` there. Creates `repo_dir` if it doesn't exist
/// yet — `repo-add` itself creates the database file on its first run, so
/// everything filesystem-side is self-healing.
Close the loop: build, sanity-check, and publish for the first time Implements the last unimplemented pipeline stage from SPEC.md: PKGBUILD generation + makepkg (builder.rs), a post-build version sanity check (sanity.rs), and repo-add publishing (publisher.rs), wired into main.rs for both the tier 1-3 auto-publish path and a new tier 4-6 review queue (`pkgwatch review` / `pkgwatch review <name> --approve`, persisted via state::{load,save,clear}_pending_version, tracked separately from last-published-version since approving one release isn't a standing auto-publish grant for future ones). Publishing targets an existing, already-registered local pacman repo (~/.local/share/pacman/custom, `[custom]` in /etc/pacman.conf) rather than one pkgwatch invents — found already in real use for a hand-packaged AppImage, which resolves SPEC's open question on where the repo lives without pkgwatch ever touching pacman.conf. Publishing stops at `repo-add`; actually installing/upgrading (`pacman -Syu`/`pacman -S`) is left to the operator, not run automatically. Getting a real second package (scaleway-cli, tier 4) through the new pipeline immediately surfaced a real gap: its pacman package is named `scaleway-cli` but the actual binary is `scw` (confirmed via `pacman -Ql` against the currently-installed extra package) — without a way to declare that, the build would install alongside extra's package under the wrong name instead of shadowing it. Added `Package::binary_name` (config.rs) to cover it. Every upstream-controlled string (version, asset name, download URL) is validated before it touches generated shell content in the PKGBUILD template — rejects anything containing a single quote or newline, since values are embedded in single-quoted bash strings. Verified for real, end to end: uv (tier 2) auto-built and published against the real astral-sh/uv release with no human step; scaleway-cli (tier 4) queued for review, then approved via `pkgwatch review scaleway-cli --approve`, which re-verified, built, and published it — confirmed the built package contains exactly usr/bin/scw. Both landed in the real custom repo's database. Left scaleway-cli's real-repo review pending rather than approving it myself: the tier 4-6 gate exists for a human judgment call, not the agent's. 69 tests, cargo make ci clean (fmt, clippy, complexity, coverage, audit). Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
2026-09-17 09:13:13 +00:00
///
/// What is *not* self-healing, and can't safely be: registering
/// `repo_name` in `/etc/pacman.conf` (see `ensure_registered`) — that
/// needs root, which this process doesn't have and shouldn't grab for
/// itself. Getting a published version onto the running system is a
/// separate, deliberate `pacman -Syu`/`pacman -S` step too, also left to
/// the operator.
Close the loop: build, sanity-check, and publish for the first time Implements the last unimplemented pipeline stage from SPEC.md: PKGBUILD generation + makepkg (builder.rs), a post-build version sanity check (sanity.rs), and repo-add publishing (publisher.rs), wired into main.rs for both the tier 1-3 auto-publish path and a new tier 4-6 review queue (`pkgwatch review` / `pkgwatch review <name> --approve`, persisted via state::{load,save,clear}_pending_version, tracked separately from last-published-version since approving one release isn't a standing auto-publish grant for future ones). Publishing targets an existing, already-registered local pacman repo (~/.local/share/pacman/custom, `[custom]` in /etc/pacman.conf) rather than one pkgwatch invents — found already in real use for a hand-packaged AppImage, which resolves SPEC's open question on where the repo lives without pkgwatch ever touching pacman.conf. Publishing stops at `repo-add`; actually installing/upgrading (`pacman -Syu`/`pacman -S`) is left to the operator, not run automatically. Getting a real second package (scaleway-cli, tier 4) through the new pipeline immediately surfaced a real gap: its pacman package is named `scaleway-cli` but the actual binary is `scw` (confirmed via `pacman -Ql` against the currently-installed extra package) — without a way to declare that, the build would install alongside extra's package under the wrong name instead of shadowing it. Added `Package::binary_name` (config.rs) to cover it. Every upstream-controlled string (version, asset name, download URL) is validated before it touches generated shell content in the PKGBUILD template — rejects anything containing a single quote or newline, since values are embedded in single-quoted bash strings. Verified for real, end to end: uv (tier 2) auto-built and published against the real astral-sh/uv release with no human step; scaleway-cli (tier 4) queued for review, then approved via `pkgwatch review scaleway-cli --approve`, which re-verified, built, and published it — confirmed the built package contains exactly usr/bin/scw. Both landed in the real custom repo's database. Left scaleway-cli's real-repo review pending rather than approving it myself: the tier 4-6 gate exists for a human judgment call, not the agent's. 69 tests, cargo make ci clean (fmt, clippy, complexity, coverage, audit). Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
2026-09-17 09:13:13 +00:00
pub fn publish(package_path: &Path, repo_dir: &Path, repo_name: &str) -> Result<PathBuf> {
publish_with(Path::new("repo-add"), package_path, repo_dir, repo_name)
}
/// `repo_add_bin` is injectable so tests can point it at a stub script
/// instead of the real `repo-add` (or a mutated global `PATH`, which would
/// race with `cargo test`'s parallel test threads).
fn publish_with(
repo_add_bin: &Path,
package_path: &Path,
repo_dir: &Path,
repo_name: &str,
) -> Result<PathBuf> {
std::fs::create_dir_all(repo_dir)
.with_context(|| format!("creating repo dir {}", repo_dir.display()))?;
let file_name = package_path
.file_name()
.context("built package path has no filename")?;
let dest = repo_dir.join(file_name);
std::fs::copy(package_path, &dest)
.with_context(|| format!("copying {} to {}", package_path.display(), dest.display()))?;
let db_path = repo_dir.join(format!("{repo_name}.db.tar.gz"));
let status = Command::new(repo_add_bin)
.arg(&db_path)
.arg(&dest)
.status()
.with_context(|| {
format!(
"running {} (is pacman-contrib installed?)",
repo_add_bin.display()
)
})?;
if !status.success() {
bail!("repo-add failed for {}", dest.display());
}
Ok(dest)
}
/// Verifies `repo_name` is registered as an active `[section]` in
/// `/etc/pacman.conf`, so a build isn't wasted on a repo pacman will never
/// actually sync from. Call this before `publish` — ideally before even
/// starting the build, so a missing repo fails fast instead of after
/// several seconds of `makepkg` work.
///
/// Doesn't check that the section's `Server =`/`Include =` line points at
/// `repo_dir` specifically — just that a repo by this name exists at all.
/// A same-named repo pointed somewhere else is a rare, easily-diagnosed
/// misconfiguration, not worth the parsing complexity to catch here.
pub fn ensure_registered(repo_name: &str, repo_dir: &Path) -> Result<()> {
ensure_registered_at(Path::new(PACMAN_CONF), repo_name, repo_dir)
}
fn ensure_registered_at(pacman_conf: &Path, repo_name: &str, repo_dir: &Path) -> Result<()> {
let conf = std::fs::read_to_string(pacman_conf)
.with_context(|| format!("reading {}", pacman_conf.display()))?;
let header = format!("[{repo_name}]");
let registered = conf.lines().map(str::trim).any(|line| line == header);
if !registered {
bail!(
"'{repo_name}' is not registered in {} — add this once, as root, then re-run:\n\n\
[{repo_name}]\n\
SigLevel = Optional TrustAll\n\
Server = file://{}\n",
pacman_conf.display(),
repo_dir.display()
);
}
Ok(())
}
Close the loop: build, sanity-check, and publish for the first time Implements the last unimplemented pipeline stage from SPEC.md: PKGBUILD generation + makepkg (builder.rs), a post-build version sanity check (sanity.rs), and repo-add publishing (publisher.rs), wired into main.rs for both the tier 1-3 auto-publish path and a new tier 4-6 review queue (`pkgwatch review` / `pkgwatch review <name> --approve`, persisted via state::{load,save,clear}_pending_version, tracked separately from last-published-version since approving one release isn't a standing auto-publish grant for future ones). Publishing targets an existing, already-registered local pacman repo (~/.local/share/pacman/custom, `[custom]` in /etc/pacman.conf) rather than one pkgwatch invents — found already in real use for a hand-packaged AppImage, which resolves SPEC's open question on where the repo lives without pkgwatch ever touching pacman.conf. Publishing stops at `repo-add`; actually installing/upgrading (`pacman -Syu`/`pacman -S`) is left to the operator, not run automatically. Getting a real second package (scaleway-cli, tier 4) through the new pipeline immediately surfaced a real gap: its pacman package is named `scaleway-cli` but the actual binary is `scw` (confirmed via `pacman -Ql` against the currently-installed extra package) — without a way to declare that, the build would install alongside extra's package under the wrong name instead of shadowing it. Added `Package::binary_name` (config.rs) to cover it. Every upstream-controlled string (version, asset name, download URL) is validated before it touches generated shell content in the PKGBUILD template — rejects anything containing a single quote or newline, since values are embedded in single-quoted bash strings. Verified for real, end to end: uv (tier 2) auto-built and published against the real astral-sh/uv release with no human step; scaleway-cli (tier 4) queued for review, then approved via `pkgwatch review scaleway-cli --approve`, which re-verified, built, and published it — confirmed the built package contains exactly usr/bin/scw. Both landed in the real custom repo's database. Left scaleway-cli's real-repo review pending rather than approving it myself: the tier 4-6 gate exists for a human judgment call, not the agent's. 69 tests, cargo make ci clean (fmt, clippy, complexity, coverage, audit). Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
2026-09-17 09:13:13 +00:00
#[cfg(test)]
mod tests {
use super::*;
fn write_stub(dir: &Path, name: &str, script: &str) -> PathBuf {
let path = dir.join(name);
std::fs::write(&path, format!("#!/bin/sh\n{script}\n")).unwrap();
let mut perms = std::fs::metadata(&path).unwrap().permissions();
std::os::unix::fs::PermissionsExt::set_mode(&mut perms, 0o755);
std::fs::set_permissions(&path, perms).unwrap();
path
}
#[test]
fn publish_copies_package_and_invokes_repo_add() {
let stub_dir = tempfile::tempdir().unwrap();
let log_path = stub_dir.path().join("invoked_with.txt");
let repo_add = write_stub(
stub_dir.path(),
"fake-repo-add",
&format!("echo \"$@\" > {}", log_path.display()),
);
let src_dir = tempfile::tempdir().unwrap();
let package_path = src_dir.path().join("thing-1.0.0-1-x86_64.pkg.tar.zst");
std::fs::write(&package_path, b"pkg-bytes").unwrap();
let repo_dir = tempfile::tempdir().unwrap();
let dest = publish_with(&repo_add, &package_path, repo_dir.path(), "custom").unwrap();
assert_eq!(
dest,
repo_dir.path().join("thing-1.0.0-1-x86_64.pkg.tar.zst")
);
assert_eq!(std::fs::read(&dest).unwrap(), b"pkg-bytes");
let invoked_with = std::fs::read_to_string(&log_path).unwrap();
assert!(invoked_with.contains("custom.db.tar.gz"));
assert!(invoked_with.contains("thing-1.0.0-1-x86_64.pkg.tar.zst"));
}
#[test]
fn publish_errors_when_repo_add_fails() {
let stub_dir = tempfile::tempdir().unwrap();
let repo_add = write_stub(stub_dir.path(), "fake-repo-add-fail", "exit 1");
let src_dir = tempfile::tempdir().unwrap();
let package_path = src_dir.path().join("thing-1.0.0-1-x86_64.pkg.tar.zst");
std::fs::write(&package_path, b"pkg-bytes").unwrap();
let repo_dir = tempfile::tempdir().unwrap();
let err = publish_with(&repo_add, &package_path, repo_dir.path(), "custom").unwrap_err();
assert!(err.to_string().contains("repo-add failed"));
}
#[test]
fn publish_creates_repo_dir_if_missing() {
let stub_dir = tempfile::tempdir().unwrap();
let repo_add = write_stub(stub_dir.path(), "fake-repo-add-ok", "exit 0");
let src_dir = tempfile::tempdir().unwrap();
let package_path = src_dir.path().join("thing-1.0.0-1-x86_64.pkg.tar.zst");
std::fs::write(&package_path, b"pkg-bytes").unwrap();
let parent = tempfile::tempdir().unwrap();
let repo_dir = parent.path().join("nested/repo");
publish_with(&repo_add, &package_path, &repo_dir, "custom").unwrap();
assert!(repo_dir.join("thing-1.0.0-1-x86_64.pkg.tar.zst").exists());
}
fn write_pacman_conf(dir: &Path, contents: &str) -> PathBuf {
let path = dir.join("pacman.conf");
std::fs::write(&path, contents).unwrap();
path
}
#[test]
fn ensure_registered_passes_when_section_present() {
let dir = tempfile::tempdir().unwrap();
let conf = write_pacman_conf(
dir.path(),
"[options]\nArchitecture = auto\n\n[extra]\nInclude = /etc/pacman.d/mirrorlist\n\n[custom]\nSigLevel = Optional TrustAll\nServer = file:///home/austin/.local/share/pacman/custom\n",
);
assert!(ensure_registered_at(&conf, "custom", Path::new("/repo")).is_ok());
}
#[test]
fn ensure_registered_fails_when_section_missing() {
let dir = tempfile::tempdir().unwrap();
let conf = write_pacman_conf(dir.path(), "[options]\nArchitecture = auto\n\n[extra]\n");
let err = ensure_registered_at(&conf, "custom", Path::new("/repo")).unwrap_err();
let msg = err.to_string();
assert!(msg.contains("not registered"));
assert!(msg.contains("[custom]"));
assert!(msg.contains("/repo"));
}
#[test]
fn ensure_registered_does_not_match_substring_of_another_section() {
// "custom" must match the whole section header, not just appear
// as a substring of e.g. "[custom-extra]".
let dir = tempfile::tempdir().unwrap();
let conf = write_pacman_conf(dir.path(), "[custom-extra]\nServer = file:///elsewhere\n");
assert!(ensure_registered_at(&conf, "custom", Path::new("/repo")).is_err());
}
#[test]
fn ensure_registered_errors_when_pacman_conf_missing() {
let dir = tempfile::tempdir().unwrap();
let missing = dir.path().join("does-not-exist.conf");
assert!(ensure_registered_at(&missing, "custom", Path::new("/repo")).is_err());
}
Close the loop: build, sanity-check, and publish for the first time Implements the last unimplemented pipeline stage from SPEC.md: PKGBUILD generation + makepkg (builder.rs), a post-build version sanity check (sanity.rs), and repo-add publishing (publisher.rs), wired into main.rs for both the tier 1-3 auto-publish path and a new tier 4-6 review queue (`pkgwatch review` / `pkgwatch review <name> --approve`, persisted via state::{load,save,clear}_pending_version, tracked separately from last-published-version since approving one release isn't a standing auto-publish grant for future ones). Publishing targets an existing, already-registered local pacman repo (~/.local/share/pacman/custom, `[custom]` in /etc/pacman.conf) rather than one pkgwatch invents — found already in real use for a hand-packaged AppImage, which resolves SPEC's open question on where the repo lives without pkgwatch ever touching pacman.conf. Publishing stops at `repo-add`; actually installing/upgrading (`pacman -Syu`/`pacman -S`) is left to the operator, not run automatically. Getting a real second package (scaleway-cli, tier 4) through the new pipeline immediately surfaced a real gap: its pacman package is named `scaleway-cli` but the actual binary is `scw` (confirmed via `pacman -Ql` against the currently-installed extra package) — without a way to declare that, the build would install alongside extra's package under the wrong name instead of shadowing it. Added `Package::binary_name` (config.rs) to cover it. Every upstream-controlled string (version, asset name, download URL) is validated before it touches generated shell content in the PKGBUILD template — rejects anything containing a single quote or newline, since values are embedded in single-quoted bash strings. Verified for real, end to end: uv (tier 2) auto-built and published against the real astral-sh/uv release with no human step; scaleway-cli (tier 4) queued for review, then approved via `pkgwatch review scaleway-cli --approve`, which re-verified, built, and published it — confirmed the built package contains exactly usr/bin/scw. Both landed in the real custom repo's database. Left scaleway-cli's real-repo review pending rather than approving it myself: the tier 4-6 gate exists for a human judgment call, not the agent's. 69 tests, cargo make ci clean (fmt, clippy, complexity, coverage, audit). Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
2026-09-17 09:13:13 +00:00
}