Close the loop: build, sanity-check, and publish for the first time
Implements the last unimplemented pipeline stage from SPEC.md: PKGBUILD
generation + makepkg (builder.rs), a post-build version sanity check
(sanity.rs), and repo-add publishing (publisher.rs), wired into main.rs
for both the tier 1-3 auto-publish path and a new tier 4-6 review queue
(`pkgwatch review` / `pkgwatch review <name> --approve`, persisted via
state::{load,save,clear}_pending_version, tracked separately from
last-published-version since approving one release isn't a standing
auto-publish grant for future ones).
Publishing targets an existing, already-registered local pacman repo
(~/.local/share/pacman/custom, `[custom]` in /etc/pacman.conf) rather
than one pkgwatch invents — found already in real use for a hand-packaged
AppImage, which resolves SPEC's open question on where the repo lives
without pkgwatch ever touching pacman.conf. Publishing stops at
`repo-add`; actually installing/upgrading (`pacman -Syu`/`pacman -S`) is
left to the operator, not run automatically.
Getting a real second package (scaleway-cli, tier 4) through the new
pipeline immediately surfaced a real gap: its pacman package is named
`scaleway-cli` but the actual binary is `scw` (confirmed via `pacman -Ql`
against the currently-installed extra package) — without a way to
declare that, the build would install alongside extra's package under
the wrong name instead of shadowing it. Added `Package::binary_name`
(config.rs) to cover it.
Every upstream-controlled string (version, asset name, download URL)
is validated before it touches generated shell content in the PKGBUILD
template — rejects anything containing a single quote or newline, since
values are embedded in single-quoted bash strings.
Verified for real, end to end: uv (tier 2) auto-built and published
against the real astral-sh/uv release with no human step; scaleway-cli
(tier 4) queued for review, then approved via `pkgwatch review
scaleway-cli --approve`, which re-verified, built, and published it —
confirmed the built package contains exactly usr/bin/scw. Both landed in
the real custom repo's database. Left scaleway-cli's real-repo review
pending rather than approving it myself: the tier 4-6 gate exists for a
human judgment call, not the agent's.
69 tests, cargo make ci clean (fmt, clippy, complexity, coverage, audit).
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
This commit is contained in:
parent
13a1381bdf
commit
68fa648010
12 changed files with 1103 additions and 113 deletions
117
SPEC.md
117
SPEC.md
|
|
@ -242,6 +242,12 @@ asset_pattern = "otherpkg-x86_64-unknown-linux-gnu.tar.gz"
|
|||
method = "same-origin-sha256"
|
||||
checksum_asset_pattern = "otherpkg-x86_64-unknown-linux-gnu.tar.gz.sha256"
|
||||
|
||||
# binary_name: only needed when the installed binary's name differs from
|
||||
# the pacman package name — e.g. real-world case, scaleway-cli's package
|
||||
# is named scaleway-cli but its actual binary is `scw` (see
|
||||
# packages.d/scaleway-cli.toml). Defaults to the package name.
|
||||
binary_name = "otherbin"
|
||||
|
||||
# Tier 1 example — not yet implemented in the PoC (only
|
||||
# same-origin-sha256 and github-attestation exist so far):
|
||||
[package.somepkg]
|
||||
|
|
@ -281,13 +287,20 @@ implements `repo`, `asset_pattern`, and `verification.method`
|
|||
against the releases API in feed order rather than trusting the first
|
||||
entry outright.
|
||||
|
||||
Tier 4-6 packages (scaleway-cli included) are not auto-published — see
|
||||
Build/publish/review-queue below. `source`, `check_method`,
|
||||
`check_interval`, and `sanity_check` are still schema sketch, not yet read
|
||||
by the code — the PoC only knows how to check GitHub-release sources.
|
||||
`sanity_check` and `binary_name` are now real, implemented fields (see
|
||||
Builder/Sanity checker above) — added `packages.d/uv.toml`'s and
|
||||
`packages.d/scaleway-cli.toml`'s own `sanity_check` blocks, and
|
||||
scaleway-cli's `binary_name = "scw"`. `source`, `check_method`, and
|
||||
`check_interval` are still schema sketch, not yet read by the code — the
|
||||
PoC only knows how to check GitHub-release sources, on a single one-shot
|
||||
run rather than a scheduled loop.
|
||||
|
||||
Build/publish/review-queue (`makepkg`, `repo-add`, tier 4–6 human review)
|
||||
are not implemented yet; a tier 4–6 pass currently just logs "flagging for
|
||||
review" and stops.
|
||||
are now implemented too — see Builder/Sanity checker/Publisher/Reviewer
|
||||
queue above and the Status entry below for the first full end-to-end run.
|
||||
Tier 4-6 packages still don't auto-publish (by design, see Verification
|
||||
trust tiers > Automation posture per tier); they queue for
|
||||
`pkgwatch review <name> --approve`.
|
||||
|
||||
Open questions on the schema:
|
||||
|
||||
|
|
@ -334,25 +347,54 @@ Open questions on the schema:
|
|||
separately. *(Implemented for `same-origin-sha256` and
|
||||
`github-attestation` — `src/verifier.rs`. The latter shells out to `gh
|
||||
attestation verify` rather than reimplementing sigstore verification.)*
|
||||
- **Builder**: for tiers 1–3 on pass, generates/updates the PKGBUILD
|
||||
(strict validation on any upstream-controlled string — version, filename
|
||||
— before it touches generated shell content; never unescaped
|
||||
interpolation) and runs `makepkg`.
|
||||
- **Builder**: for tiers 1–3 on pass, generates a PKGBUILD (strict
|
||||
validation on every upstream-controlled string — version, asset name,
|
||||
download URL — before it touches generated shell content; every
|
||||
interpolated value is embedded in a single-quoted bash string and a
|
||||
literal `'` or newline in the input is rejected outright, never
|
||||
unescaped interpolation) and runs `makepkg`. *(Implemented —
|
||||
`src/builder.rs`. One fixed "prebuilt binary" PKGBUILD shape covers both
|
||||
tracked packages so far: a bare-binary download (scaleway-cli) and a
|
||||
tarball extracting to a same-named directory (uv) — see Scaling >
|
||||
Template reuse. `Package.binary_name` (config.rs) covers the case where
|
||||
the installed binary's name differs from the pacman package name, which
|
||||
turned out to matter immediately: scaleway-cli's real binary is `scw`,
|
||||
not `scaleway-cli` — confirmed by inspecting the currently-installed
|
||||
`extra` package with `pacman -Ql`, not guessable from the repo name.
|
||||
Without it the build would install alongside `extra`'s package instead
|
||||
of shadowing it.)*
|
||||
- **Sanity checker**: after a successful build, runs the package's
|
||||
declared `sanity_check.command` against the built artifact and confirms
|
||||
the reported version matches what pkgwatch believes it just built.
|
||||
Mismatch = fail loud, do not publish. This is a correctness check, not a
|
||||
security control — it catches checker bugs and mangled/wrong-artifact
|
||||
downloads, not malicious releases.
|
||||
- **Publisher**: runs `repo-add` against the local repo, only after the
|
||||
sanity check passes.
|
||||
declared `sanity_check.command` — with the freshly built package's
|
||||
`usr/bin` prepended to `PATH`, so it exercises what was just built
|
||||
rather than whatever's already installed system-wide — and confirms the
|
||||
reported version matches what pkgwatch believes it just built. Mismatch
|
||||
= fail loud, do not publish. This is a correctness check, not a security
|
||||
control — it catches checker bugs and mangled/wrong-artifact downloads,
|
||||
not malicious releases. *(Implemented — `src/sanity.rs`.)*
|
||||
- **Publisher**: copies the built package into the local repo directory
|
||||
and runs `repo-add`, only after the sanity check passes. *(Implemented —
|
||||
`src/publisher.rs`. Targets an existing, already-registered local pacman
|
||||
repo rather than one pkgwatch creates — this box already has one at
|
||||
`~/.local/share/pacman/custom`, registered as `[custom]` in
|
||||
`/etc/pacman.conf` (`SigLevel = Optional TrustAll`) and already in use
|
||||
for a hand-packaged AppImage, resolving the "where does the repo live /
|
||||
how does it get registered" open question below without pkgwatch ever
|
||||
touching pacman.conf. Deliberately stops at `repo-add`: getting the new
|
||||
version onto the running system is a separate, deliberate
|
||||
`pacman -Syu`/`pacman -S <pkg>` step left to the operator, not run
|
||||
automatically.)*
|
||||
- **Reviewer queue**: for tiers 4–6, records the detected change instead of
|
||||
auto-building; a separate `pkgwatch review` command lets a human
|
||||
approve/reject, which then triggers the build → sanity-check → publish
|
||||
steps above.
|
||||
steps above. *(Implemented — `state::{load,save,clear}_pending_version`
|
||||
plus the `review`/`review <name> --approve` subcommands in `src/main.rs`.
|
||||
Tracked separately from the last-published-version state: approving one
|
||||
release doesn't mean future ones auto-publish. `--approve` re-verifies
|
||||
before building rather than trusting a possibly-stale flag from an
|
||||
earlier run. No reject/dismiss command yet — see Status below.)*
|
||||
- **Scheduling**: systemd `.service` (oneshot) + `.timer` running it
|
||||
periodically, matching the pattern already used for other periodic tasks
|
||||
on this box.
|
||||
on this box. *(Not implemented — still a single one-shot `cargo run`.)*
|
||||
|
||||
## Prior art / reference points
|
||||
|
||||
|
|
@ -395,18 +437,33 @@ Open questions on the schema:
|
|||
repo's newest feed entry, a `-dbg1` tag, has no real Release behind
|
||||
it). Still just flags for human review, same as any tier 4-6 pass —
|
||||
not auto-installed; see the unchecked build/publish item below.
|
||||
- [ ] Not yet implemented: build (PKGBUILD generation + `makepkg`),
|
||||
publish (`repo-add`), reviewer queue for tier 4–6, scheduling/
|
||||
`check_interval`, non-GitHub sources, `minisign`/tier-1 method.
|
||||
- [ ] Refine config schema further (see open questions above), including
|
||||
the `sanity_check` block per package.
|
||||
- [x] Full pipeline closed end to end for the first time: check → fetch →
|
||||
verify → build → sanity-check → publish, against two real packages.
|
||||
`uv` (tier 2) auto-built and published on the first run with no
|
||||
human step. `scaleway-cli` (tier 4) queued for review, then
|
||||
`pkgwatch review scaleway-cli --approve` re-verified, built, and
|
||||
published it — confirmed the built package installs as
|
||||
`/usr/bin/scw`, actually shadowing `extra`'s package rather than
|
||||
installing alongside it under the wrong name. Both landed in the
|
||||
real `~/.local/share/pacman/custom` repo's database
|
||||
(`custom.db.tar.gz`), ready for `sudo pacman -Syu`/`sudo pacman -S`
|
||||
— not run automatically. See Builder/Sanity checker/Publisher/
|
||||
Reviewer queue above for what each piece does.
|
||||
One cosmetic wrinkle, not a correctness issue: `makepkg` printed
|
||||
`libfakeroot internal error: payload not recognized!` while
|
||||
packaging scaleway-cli's large Go binary, but still produced a
|
||||
correct package (verified: exactly `usr/bin/scw` plus standard
|
||||
metadata) — looks like an environment quirk in this sandbox's
|
||||
fakeroot, not something pkgwatch caused; revisit if a real build
|
||||
ever actually fails on it.
|
||||
- [ ] Not yet implemented: `pkgwatch review <name> --reject` (a pending
|
||||
review can only be approved or left pending, not dismissed),
|
||||
scheduling/`check_interval`, non-GitHub sources, `minisign`/tier-1
|
||||
method, retention/pruning of old versions in the local repo (see
|
||||
Scaling > Local repo retention), staggering/auth for GitHub API
|
||||
rate limits at higher package counts.
|
||||
- [ ] Refine config schema further (see open questions above).
|
||||
- [ ] Decide version-check strategy for non-GitHub sources: shell out to
|
||||
`nvchecker` vs. own implementation.
|
||||
- [ ] Implement PKGBUILD generation with strict upstream-string validation
|
||||
from day one (see Builder, above) — cheap to do right up front,
|
||||
expensive to retrofit.
|
||||
- [ ] Next PoC iteration: carry the verified `uv` artifact through
|
||||
build → sanity-check → `repo-add` publish, closing the loop to an
|
||||
actual local pacman repo `pacman -Syu` can pick up.
|
||||
- [ ] Decide on project home: local-only for now, or push to
|
||||
code.austinschaefer.com (Forgejo) once the spec settles.
|
||||
|
|
|
|||
|
|
@ -7,11 +7,22 @@
|
|||
# Releases ship one combined `SHA256SUMS` file (one line per platform
|
||||
# asset) rather than a per-asset checksum file like uv's — verifier
|
||||
# matches the line by filename.
|
||||
#
|
||||
# binary_name = "scw": confirmed by checking the currently-installed
|
||||
# `extra` package (`pacman -Ql scaleway-cli`) — the pacman package is
|
||||
# named scaleway-cli but the actual binary it installs is `scw`. Without
|
||||
# this, pkgwatch's build would install as /usr/bin/scaleway-cli, which
|
||||
# would NOT shadow extra's /usr/bin/scw at all.
|
||||
|
||||
[package.scaleway-cli]
|
||||
repo = "scaleway/scaleway-cli"
|
||||
asset_pattern = "scaleway-cli_{version}_linux_amd64"
|
||||
binary_name = "scw"
|
||||
|
||||
[package.scaleway-cli.verification]
|
||||
method = "same-origin-sha256"
|
||||
checksum_asset_pattern = "SHA256SUMS"
|
||||
|
||||
[package.scaleway-cli.sanity_check]
|
||||
command = "scw version"
|
||||
version_regex = 'Version\s+(\d+\.\d+\.\d+)'
|
||||
|
|
|
|||
|
|
@ -10,3 +10,7 @@ asset_pattern = "uv-x86_64-unknown-linux-gnu.tar.gz"
|
|||
|
||||
[package.uv.verification]
|
||||
method = "github-attestation"
|
||||
|
||||
[package.uv.sanity_check]
|
||||
command = "uv --version"
|
||||
version_regex = 'uv (\d+\.\d+\.\d+)'
|
||||
|
|
|
|||
359
src/builder.rs
Normal file
359
src/builder.rs
Normal file
|
|
@ -0,0 +1,359 @@
|
|||
use crate::config::Package;
|
||||
use crate::hash::sha256_hex;
|
||||
use anyhow::{Context, Result, bail};
|
||||
use std::path::{Path, PathBuf};
|
||||
use std::process::Command;
|
||||
|
||||
/// Archive extensions `makepkg` auto-extracts before `package()` runs.
|
||||
/// Longest-first so `.tar.gz` isn't shadowed by a hypothetical `.gz` entry.
|
||||
const ARCHIVE_EXTENSIONS: &[&str] = &[".tar.gz", ".tar.xz", ".tar.zst", ".tar.bz2", ".tgz", ".zip"];
|
||||
|
||||
/// Everything needed to generate and build a PKGBUILD for one release.
|
||||
pub struct BuildRequest<'a> {
|
||||
pub pkg_name: &'a str,
|
||||
pub pkg: &'a Package,
|
||||
pub version: &'a str,
|
||||
pub repo: &'a str,
|
||||
pub asset_name: &'a str,
|
||||
pub download_url: &'a str,
|
||||
pub artifact_path: &'a Path,
|
||||
}
|
||||
|
||||
pub struct BuildResult {
|
||||
/// The built `.pkg.tar.zst`, ready for `publisher::publish`.
|
||||
pub package_path: PathBuf,
|
||||
/// `makepkg`'s package staging directory (`$pkgdir`), still present
|
||||
/// after a successful build — lets `sanity` exercise the freshly built
|
||||
/// binary without installing it system-wide first.
|
||||
pub pkgdir: PathBuf,
|
||||
}
|
||||
|
||||
/// Generates a PKGBUILD around an already-downloaded, already-verified
|
||||
/// artifact, then runs `makepkg` in `build_dir`.
|
||||
///
|
||||
/// Deliberately one fixed "prebuilt binary" shape, not a templating engine
|
||||
/// — see SPEC.md > Scaling > Template reuse. Covers the two shapes the two
|
||||
/// currently-tracked packages actually need: a bare-binary download
|
||||
/// (scaleway-cli) and a tarball containing a same-named directory (uv).
|
||||
/// Extend when a third real shape shows up rather than guessing at
|
||||
/// generality now.
|
||||
pub fn build(req: &BuildRequest, build_dir: &Path) -> Result<BuildResult> {
|
||||
let pkgbuild = generate_pkgbuild(req)?;
|
||||
|
||||
std::fs::create_dir_all(build_dir)
|
||||
.with_context(|| format!("creating build dir {}", build_dir.display()))?;
|
||||
std::fs::write(build_dir.join("PKGBUILD"), pkgbuild)?;
|
||||
// makepkg looks for the source file by its declared name next to
|
||||
// PKGBUILD; pre-seed it with the copy pkgwatch already downloaded and
|
||||
// verified so makepkg's own sha256 check passes without re-fetching
|
||||
// from the network (and without trusting the network a second time).
|
||||
std::fs::copy(req.artifact_path, build_dir.join(req.asset_name))?;
|
||||
|
||||
let status = Command::new("makepkg")
|
||||
.args(["--noconfirm", "--force"])
|
||||
.current_dir(build_dir)
|
||||
.status()
|
||||
.context("running makepkg (is base-devel installed?)")?;
|
||||
if !status.success() {
|
||||
bail!("makepkg failed for {} {}", req.pkg_name, req.version);
|
||||
}
|
||||
|
||||
let package_path = find_built_package(build_dir, req.pkg_name, req.version)?;
|
||||
let pkgdir = build_dir.join("pkg").join(req.pkg_name);
|
||||
Ok(BuildResult {
|
||||
package_path,
|
||||
pkgdir,
|
||||
})
|
||||
}
|
||||
|
||||
/// Builds the PKGBUILD text for `req`, validating every upstream-controlled
|
||||
/// string first (see SPEC.md > Architecture > Builder: "strict validation
|
||||
/// on any upstream-controlled string ... never unescaped interpolation").
|
||||
/// Pure and side-effect-free so it's testable without invoking `makepkg`.
|
||||
fn generate_pkgbuild(req: &BuildRequest) -> Result<String> {
|
||||
validate_pkgname(req.pkg_name)?;
|
||||
validate_pkgver(req.version)?;
|
||||
validate_shell_safe("asset name", req.asset_name)?;
|
||||
validate_shell_safe("download url", req.download_url)?;
|
||||
validate_shell_safe("repo", req.repo)?;
|
||||
|
||||
let binary_name = req.pkg.binary_name(req.pkg_name);
|
||||
validate_pkgname(binary_name)?;
|
||||
|
||||
let artifact_data = std::fs::read(req.artifact_path)
|
||||
.with_context(|| format!("reading {}", req.artifact_path.display()))?;
|
||||
let sha256 = sha256_hex(&artifact_data);
|
||||
|
||||
let install_source = match archive_stem(req.asset_name) {
|
||||
Some(stem) => format!("{stem}/{binary_name}"),
|
||||
None => req.asset_name.to_string(),
|
||||
};
|
||||
|
||||
Ok(format!(
|
||||
"# Maintainer: pkgwatch (auto-generated — do not edit by hand,\n\
|
||||
# edits are overwritten on the next update)\n\
|
||||
pkgname='{name}'\n\
|
||||
pkgver='{version}'\n\
|
||||
pkgrel=1\n\
|
||||
pkgdesc='{repo} release {version}, packaged by pkgwatch'\n\
|
||||
arch=('x86_64')\n\
|
||||
url='https://github.com/{repo}'\n\
|
||||
license=('unknown')\n\
|
||||
options=('!strip')\n\
|
||||
source=('{asset}::{url}')\n\
|
||||
sha256sums=('{sha256}')\n\
|
||||
\n\
|
||||
package() {{\n\
|
||||
\x20 install -Dm755 \"${{srcdir}}/{install_source}\" \"${{pkgdir}}/usr/bin/{binary_name}\"\n\
|
||||
}}\n",
|
||||
name = req.pkg_name,
|
||||
version = req.version,
|
||||
repo = req.repo,
|
||||
asset = req.asset_name,
|
||||
url = req.download_url,
|
||||
))
|
||||
}
|
||||
|
||||
fn find_built_package(build_dir: &Path, pkg_name: &str, version: &str) -> Result<PathBuf> {
|
||||
let prefix = format!("{pkg_name}-{version}-");
|
||||
for entry in std::fs::read_dir(build_dir)? {
|
||||
let path = entry?.path();
|
||||
let Some(file_name) = path.file_name().and_then(|n| n.to_str()) else {
|
||||
continue;
|
||||
};
|
||||
if file_name.starts_with(&prefix) && file_name.ends_with(".pkg.tar.zst") {
|
||||
return Ok(path);
|
||||
}
|
||||
}
|
||||
bail!(
|
||||
"makepkg reported success but no {prefix}*.pkg.tar.zst found in {}",
|
||||
build_dir.display()
|
||||
)
|
||||
}
|
||||
|
||||
/// Strips a recognized archive extension, returning the resulting stem —
|
||||
/// the directory name `makepkg` extracts a same-named tarball into, by
|
||||
/// the convention every currently-tracked tarball-shaped package follows.
|
||||
/// `None` means the asset is a bare binary download (no extraction).
|
||||
fn archive_stem(asset_name: &str) -> Option<&str> {
|
||||
ARCHIVE_EXTENSIONS
|
||||
.iter()
|
||||
.find_map(|ext| asset_name.strip_suffix(ext))
|
||||
}
|
||||
|
||||
/// Rejects a single quote or newline: both would let upstream-controlled
|
||||
/// text (asset names, download URLs) break out of the single-quoted bash
|
||||
/// strings the PKGBUILD template embeds them in. See SPEC.md > Architecture
|
||||
/// > Builder ("never unescaped interpolation").
|
||||
fn validate_shell_safe(field: &str, value: &str) -> Result<()> {
|
||||
if value.contains('\'') || value.contains('\n') {
|
||||
bail!("{field} '{value}' contains an unsafe character for a generated PKGBUILD");
|
||||
}
|
||||
Ok(())
|
||||
}
|
||||
|
||||
/// A pacman `pkgver` may only contain alphanumerics, `.`, `_`, `+` — no
|
||||
/// hyphens (pacman reserves `-` as the pkgver/pkgrel separator in the
|
||||
/// final package filename) and no shell metacharacters.
|
||||
fn validate_pkgver(version: &str) -> Result<()> {
|
||||
let valid = !version.is_empty()
|
||||
&& version
|
||||
.chars()
|
||||
.all(|c| c.is_ascii_alphanumeric() || matches!(c, '.' | '_' | '+'));
|
||||
if !valid {
|
||||
bail!("'{version}' is not a valid pacman pkgver (only [A-Za-z0-9._+] allowed)");
|
||||
}
|
||||
Ok(())
|
||||
}
|
||||
|
||||
/// A pacman package/binary name may only contain lowercase alphanumerics
|
||||
/// plus `@ . _ + -`.
|
||||
fn validate_pkgname(name: &str) -> Result<()> {
|
||||
let valid = !name.is_empty()
|
||||
&& name.chars().all(|c| {
|
||||
c.is_ascii_lowercase() || c.is_ascii_digit() || matches!(c, '@' | '.' | '_' | '+' | '-')
|
||||
});
|
||||
if !valid {
|
||||
bail!("'{name}' is not a valid pacman package name");
|
||||
}
|
||||
Ok(())
|
||||
}
|
||||
|
||||
#[cfg(test)]
|
||||
mod tests {
|
||||
use super::*;
|
||||
|
||||
#[test]
|
||||
fn archive_stem_strips_known_extensions() {
|
||||
assert_eq!(
|
||||
archive_stem("uv-x86_64-unknown-linux-gnu.tar.gz"),
|
||||
Some("uv-x86_64-unknown-linux-gnu")
|
||||
);
|
||||
assert_eq!(archive_stem("thing.zip"), Some("thing"));
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn archive_stem_none_for_bare_binary() {
|
||||
assert_eq!(archive_stem("scaleway-cli_2.62.0_linux_amd64"), None);
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn validate_pkgver_accepts_dotted_version() {
|
||||
assert!(validate_pkgver("2.62.0").is_ok());
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn validate_pkgver_rejects_hyphen() {
|
||||
assert!(validate_pkgver("2.62.0-dbg1").is_err());
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn validate_pkgver_rejects_shell_metacharacters() {
|
||||
assert!(validate_pkgver("2.62.0; rm -rf /").is_err());
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn validate_pkgver_rejects_empty() {
|
||||
assert!(validate_pkgver("").is_err());
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn validate_pkgname_accepts_hyphenated_name() {
|
||||
assert!(validate_pkgname("scaleway-cli").is_ok());
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn validate_pkgname_rejects_uppercase() {
|
||||
assert!(validate_pkgname("Scaleway-CLI").is_err());
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn validate_shell_safe_rejects_single_quote() {
|
||||
assert!(validate_shell_safe("asset name", "thing'; touch pwned #.tar.gz").is_err());
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn validate_shell_safe_rejects_newline() {
|
||||
assert!(validate_shell_safe("download url", "https://example.com/a\nb").is_err());
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn validate_shell_safe_accepts_normal_url() {
|
||||
assert!(validate_shell_safe("download url", "https://example.com/a/b.tar.gz").is_ok());
|
||||
}
|
||||
|
||||
fn make_package(binary_name: Option<&str>) -> Package {
|
||||
let toml_text = match binary_name {
|
||||
Some(bin) => format!(
|
||||
r#"
|
||||
repo = "o/r"
|
||||
asset_pattern = "x"
|
||||
binary_name = "{bin}"
|
||||
[verification]
|
||||
method = "github-attestation"
|
||||
"#
|
||||
),
|
||||
None => r#"
|
||||
repo = "o/r"
|
||||
asset_pattern = "x"
|
||||
[verification]
|
||||
method = "github-attestation"
|
||||
"#
|
||||
.to_string(),
|
||||
};
|
||||
toml::from_str(&toml_text).unwrap()
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn build_rejects_unsafe_version() {
|
||||
let pkg = make_package(None);
|
||||
let dir = tempfile::tempdir().unwrap();
|
||||
let artifact_path = dir.path().join("thing.tar.gz");
|
||||
std::fs::write(&artifact_path, b"data").unwrap();
|
||||
|
||||
let req = BuildRequest {
|
||||
pkg_name: "thing",
|
||||
pkg: &pkg,
|
||||
version: "1.0.0-dbg1",
|
||||
repo: "o/r",
|
||||
asset_name: "thing.tar.gz",
|
||||
download_url: "https://example.com/thing.tar.gz",
|
||||
artifact_path: &artifact_path,
|
||||
};
|
||||
let build_dir = dir.path().join("build");
|
||||
assert!(build(&req, &build_dir).is_err());
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn generate_pkgbuild_bare_binary_installs_under_binary_name_override() {
|
||||
let pkg = make_package(Some("scw"));
|
||||
let dir = tempfile::tempdir().unwrap();
|
||||
let artifact_path = dir.path().join("scaleway-cli_2.62.0_linux_amd64");
|
||||
std::fs::write(&artifact_path, b"binary-bytes").unwrap();
|
||||
let expected_sha = sha256_hex(b"binary-bytes");
|
||||
|
||||
let req = BuildRequest {
|
||||
pkg_name: "scaleway-cli",
|
||||
pkg: &pkg,
|
||||
version: "2.62.0",
|
||||
repo: "scaleway/scaleway-cli",
|
||||
asset_name: "scaleway-cli_2.62.0_linux_amd64",
|
||||
download_url: "https://github.com/scaleway/scaleway-cli/releases/download/v2.62.0/scaleway-cli_2.62.0_linux_amd64",
|
||||
artifact_path: &artifact_path,
|
||||
};
|
||||
let pkgbuild = generate_pkgbuild(&req).unwrap();
|
||||
|
||||
assert!(pkgbuild.contains("pkgname='scaleway-cli'"));
|
||||
assert!(pkgbuild.contains("pkgver='2.62.0'"));
|
||||
assert!(pkgbuild.contains(&format!("sha256sums=('{expected_sha}')")));
|
||||
// Bare binary (no archive extension) — installed straight from
|
||||
// srcdir under the overridden binary name, not the pkgname.
|
||||
assert!(pkgbuild.contains(
|
||||
"install -Dm755 \"${srcdir}/scaleway-cli_2.62.0_linux_amd64\" \"${pkgdir}/usr/bin/scw\""
|
||||
));
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn generate_pkgbuild_tarball_installs_from_extracted_stem_dir() {
|
||||
let pkg = make_package(None);
|
||||
let dir = tempfile::tempdir().unwrap();
|
||||
let artifact_path = dir.path().join("uv-x86_64-unknown-linux-gnu.tar.gz");
|
||||
std::fs::write(&artifact_path, b"tarball-bytes").unwrap();
|
||||
|
||||
let req = BuildRequest {
|
||||
pkg_name: "uv",
|
||||
pkg: &pkg,
|
||||
version: "0.12.15",
|
||||
repo: "astral-sh/uv",
|
||||
asset_name: "uv-x86_64-unknown-linux-gnu.tar.gz",
|
||||
download_url: "https://github.com/astral-sh/uv/releases/download/0.12.15/uv-x86_64-unknown-linux-gnu.tar.gz",
|
||||
artifact_path: &artifact_path,
|
||||
};
|
||||
let pkgbuild = generate_pkgbuild(&req).unwrap();
|
||||
|
||||
// No binary_name override — pkgname doubles as the binary name,
|
||||
// and makepkg extracts the tarball into a same-named directory.
|
||||
assert!(pkgbuild.contains(
|
||||
"install -Dm755 \"${srcdir}/uv-x86_64-unknown-linux-gnu/uv\" \"${pkgdir}/usr/bin/uv\""
|
||||
));
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn generate_pkgbuild_rejects_download_url_with_single_quote() {
|
||||
let pkg = make_package(None);
|
||||
let dir = tempfile::tempdir().unwrap();
|
||||
let artifact_path = dir.path().join("thing.tar.gz");
|
||||
std::fs::write(&artifact_path, b"data").unwrap();
|
||||
|
||||
let req = BuildRequest {
|
||||
pkg_name: "thing",
|
||||
pkg: &pkg,
|
||||
version: "1.0.0",
|
||||
repo: "o/r",
|
||||
asset_name: "thing.tar.gz",
|
||||
download_url: "https://example.com/x'; touch pwned #.tar.gz",
|
||||
artifact_path: &artifact_path,
|
||||
};
|
||||
assert!(generate_pkgbuild(&req).is_err());
|
||||
}
|
||||
}
|
||||
|
|
@ -18,6 +18,31 @@ pub struct Package {
|
|||
/// `checker::version_from_tag` before matching.
|
||||
pub asset_pattern: String,
|
||||
pub verification: Verification,
|
||||
/// Name of the executable inside the built package, if it differs from
|
||||
/// the package name itself — e.g. scaleway-cli's pacman package is
|
||||
/// named `scaleway-cli` but its real binary is `scw` (discovered by
|
||||
/// checking the currently-installed extra package, not guessable from
|
||||
/// the repo name). Defaults to the package name when omitted.
|
||||
pub binary_name: Option<String>,
|
||||
/// Post-build correctness check (not a security control — see
|
||||
/// SPEC.md > Verification trust tiers). Runs `command` against the
|
||||
/// freshly built binary and confirms `version_regex`'s capture group
|
||||
/// matches the version pkgwatch believes it just built.
|
||||
pub sanity_check: Option<SanityCheck>,
|
||||
}
|
||||
|
||||
impl Package {
|
||||
/// The name of the executable inside the built package: `binary_name`
|
||||
/// if the package declares one, else `pkg_name` itself.
|
||||
pub fn binary_name<'a>(&'a self, pkg_name: &'a str) -> &'a str {
|
||||
self.binary_name.as_deref().unwrap_or(pkg_name)
|
||||
}
|
||||
}
|
||||
|
||||
#[derive(Debug, Deserialize, Clone)]
|
||||
pub struct SanityCheck {
|
||||
pub command: String,
|
||||
pub version_regex: String,
|
||||
}
|
||||
|
||||
#[derive(Debug, Deserialize, Clone)]
|
||||
|
|
|
|||
|
|
@ -14,8 +14,18 @@ struct Asset {
|
|||
browser_download_url: String,
|
||||
}
|
||||
|
||||
/// A downloaded release asset: its local path plus the URL it came from,
|
||||
/// the latter needed for the `source=` line of a generated PKGBUILD (see
|
||||
/// `builder`) — `makepkg` uses it only as a fallback if the pre-seeded
|
||||
/// local copy ever goes missing.
|
||||
#[derive(Debug, Clone, PartialEq, Eq)]
|
||||
pub struct DownloadedAsset {
|
||||
pub path: PathBuf,
|
||||
pub download_url: String,
|
||||
}
|
||||
|
||||
/// Downloads the release asset named exactly `asset_name` for `repo`@`tag`
|
||||
/// into `dest_dir`, returning the local path.
|
||||
/// into `dest_dir`, returning the local path and its origin URL.
|
||||
pub fn download_asset(
|
||||
client: &reqwest::blocking::Client,
|
||||
endpoints: &GithubEndpoints,
|
||||
|
|
@ -23,7 +33,7 @@ pub fn download_asset(
|
|||
tag: &str,
|
||||
asset_name: &str,
|
||||
dest_dir: &Path,
|
||||
) -> Result<PathBuf> {
|
||||
) -> Result<DownloadedAsset> {
|
||||
let api_url = format!("{}/repos/{repo}/releases/tags/{tag}", endpoints.api);
|
||||
let release: Release = client
|
||||
.get(&api_url)
|
||||
|
|
@ -46,7 +56,10 @@ pub fn download_asset(
|
|||
.error_for_status()?
|
||||
.bytes()?;
|
||||
std::fs::write(&dest_path, &bytes)?;
|
||||
Ok(dest_path)
|
||||
Ok(DownloadedAsset {
|
||||
path: dest_path,
|
||||
download_url: asset.browser_download_url.clone(),
|
||||
})
|
||||
}
|
||||
|
||||
#[cfg(test)]
|
||||
|
|
@ -77,7 +90,7 @@ mod tests {
|
|||
|
||||
let client = reqwest::blocking::Client::new();
|
||||
let dest_dir = tempfile::tempdir().unwrap();
|
||||
let path = download_asset(
|
||||
let asset = download_asset(
|
||||
&client,
|
||||
&endpoints,
|
||||
"o/r",
|
||||
|
|
@ -87,8 +100,9 @@ mod tests {
|
|||
)
|
||||
.unwrap();
|
||||
|
||||
assert_eq!(path, dest_dir.path().join("thing.tar.gz"));
|
||||
assert_eq!(std::fs::read(&path).unwrap(), b"artifact-bytes");
|
||||
assert_eq!(asset.path, dest_dir.path().join("thing.tar.gz"));
|
||||
assert_eq!(asset.download_url, asset_url);
|
||||
assert_eq!(std::fs::read(&asset.path).unwrap(), b"artifact-bytes");
|
||||
}
|
||||
|
||||
#[test]
|
||||
|
|
|
|||
25
src/hash.rs
Normal file
25
src/hash.rs
Normal file
|
|
@ -0,0 +1,25 @@
|
|||
use sha2::{Digest, Sha256};
|
||||
|
||||
/// Shared by `verifier` (same-origin-sha256 checks) and `builder` (every
|
||||
/// generated PKGBUILD needs a `sha256sums` entry for makepkg's own local
|
||||
/// integrity check, regardless of pkgwatch's own trust tier for that
|
||||
/// package).
|
||||
pub fn sha256_hex(data: &[u8]) -> String {
|
||||
let mut hasher = Sha256::new();
|
||||
hasher.update(data);
|
||||
hex::encode(hasher.finalize())
|
||||
}
|
||||
|
||||
#[cfg(test)]
|
||||
mod tests {
|
||||
use super::*;
|
||||
|
||||
#[test]
|
||||
fn matches_known_sha256() {
|
||||
// printf 'hello world' | sha256sum
|
||||
assert_eq!(
|
||||
sha256_hex(b"hello world"),
|
||||
"b94d27b9934d3e08a52e52d7da7dabfac484efe37a5380ee9088f7ace2efcde9"
|
||||
);
|
||||
}
|
||||
}
|
||||
317
src/main.rs
317
src/main.rs
|
|
@ -1,25 +1,65 @@
|
|||
mod builder;
|
||||
mod checker;
|
||||
mod config;
|
||||
mod fetcher;
|
||||
mod github;
|
||||
mod hash;
|
||||
mod publisher;
|
||||
mod sanity;
|
||||
mod state;
|
||||
mod verifier;
|
||||
|
||||
use anyhow::Result;
|
||||
use anyhow::{Context, Result, bail};
|
||||
use config::Package;
|
||||
use fetcher::DownloadedAsset;
|
||||
use github::GithubEndpoints;
|
||||
use std::path::Path;
|
||||
use std::path::{Path, PathBuf};
|
||||
use verifier::VerificationResult;
|
||||
|
||||
/// First iteration: check -> fetch -> verify -> report, for whatever is
|
||||
/// in packages.d/. No build/publish step yet (see SPEC.md > Status).
|
||||
const PACKAGES_DIR: &str = "packages.d";
|
||||
const STATE_DIR: &str = "state";
|
||||
const WORK_DIR: &str = "work";
|
||||
/// Not a repo pkgwatch invents: this is the existing, already-registered
|
||||
/// local pacman repo on this box (see `[custom]` in /etc/pacman.conf and
|
||||
/// its `Server = file://...` line). pkgwatch adds packages to it; it does
|
||||
/// not create the repo or touch pacman.conf.
|
||||
const CUSTOM_REPO_NAME: &str = "custom";
|
||||
const CUSTOM_REPO_SUBPATH: &str = ".local/share/pacman/custom";
|
||||
|
||||
/// check -> fetch -> verify -> build -> sanity-check -> publish, for
|
||||
/// whatever is in packages.d/. Tier 1-3 passes auto-publish; tier 4-6
|
||||
/// passes queue for `pkgwatch review`. See SPEC.md > Architecture.
|
||||
fn main() -> Result<()> {
|
||||
let client = reqwest::blocking::Client::builder()
|
||||
.user_agent("pkgwatch/0.1 (PoC; https://code.austinschaefer.com)")
|
||||
.build()?;
|
||||
let endpoints = GithubEndpoints::default();
|
||||
let args: Vec<String> = std::env::args().skip(1).collect();
|
||||
match args.first().map(String::as_str) {
|
||||
None => run_check(),
|
||||
Some("review") => run_review(&args[1..]),
|
||||
Some(other) => bail!("unknown subcommand '{other}' (expected: review)"),
|
||||
}
|
||||
}
|
||||
|
||||
let packages_dir = Path::new("packages.d");
|
||||
let state_dir = Path::new("state");
|
||||
let work_dir = Path::new("work");
|
||||
fn build_client() -> Result<reqwest::blocking::Client> {
|
||||
Ok(reqwest::blocking::Client::builder()
|
||||
.user_agent("pkgwatch/0.1 (PoC; https://code.austinschaefer.com)")
|
||||
.build()?)
|
||||
}
|
||||
|
||||
fn custom_repo_dir() -> Result<PathBuf> {
|
||||
// Override for testing against a scratch repo instead of the real one
|
||||
// at $HOME/.local/share/pacman/custom.
|
||||
if let Ok(dir) = std::env::var("PKGWATCH_REPO_DIR") {
|
||||
return Ok(PathBuf::from(dir));
|
||||
}
|
||||
let home = std::env::var("HOME").context("HOME is not set")?;
|
||||
Ok(Path::new(&home).join(CUSTOM_REPO_SUBPATH))
|
||||
}
|
||||
|
||||
fn run_check() -> Result<()> {
|
||||
let client = build_client()?;
|
||||
let endpoints = GithubEndpoints::default();
|
||||
let packages_dir = Path::new(PACKAGES_DIR);
|
||||
let state_dir = Path::new(STATE_DIR);
|
||||
let work_dir = Path::new(WORK_DIR);
|
||||
|
||||
let packages = config::load_packages_dir(packages_dir)?;
|
||||
if packages.is_empty() {
|
||||
|
|
@ -27,66 +67,207 @@ fn main() -> Result<()> {
|
|||
return Ok(());
|
||||
}
|
||||
|
||||
for (name, pkg) in packages {
|
||||
let mut any_failed = false;
|
||||
for (name, pkg) in &packages {
|
||||
println!("== {name} ({}) ==", pkg.repo);
|
||||
|
||||
let latest = checker::latest_github_release(&client, &endpoints, &pkg.repo)?;
|
||||
let last_seen = state::load_last_version(state_dir, &name);
|
||||
|
||||
if last_seen.as_deref() == Some(latest.as_str()) {
|
||||
println!(" up to date at {latest}");
|
||||
continue;
|
||||
}
|
||||
|
||||
println!(" new version detected: {latest} (previously: {last_seen:?})");
|
||||
|
||||
let dest_dir = work_dir.join(&name).join(&latest);
|
||||
let asset_name = pkg
|
||||
.asset_pattern
|
||||
.replace("{version}", checker::version_from_tag(&latest));
|
||||
let artifact_path = fetcher::download_asset(
|
||||
&client,
|
||||
&endpoints,
|
||||
&pkg.repo,
|
||||
&latest,
|
||||
&asset_name,
|
||||
&dest_dir,
|
||||
)?;
|
||||
println!(" fetched {}", artifact_path.display());
|
||||
|
||||
let result = verifier::verify(
|
||||
&client,
|
||||
&endpoints,
|
||||
&pkg.verification,
|
||||
&pkg.repo,
|
||||
&latest,
|
||||
&artifact_path,
|
||||
&dest_dir,
|
||||
)?;
|
||||
|
||||
println!(
|
||||
" verification (tier {}): {} — {}",
|
||||
result.tier,
|
||||
if result.passed { "PASS" } else { "FAIL" },
|
||||
result.justification
|
||||
);
|
||||
|
||||
match (result.tier, result.passed) {
|
||||
(1..=3, true) => {
|
||||
println!(" tier 1-3 pass: would auto-build + publish (not yet implemented)");
|
||||
state::save_last_version(state_dir, &name, &latest)?;
|
||||
}
|
||||
(_, true) => {
|
||||
println!(" tier 4-6 pass: flagging for human review, not auto-publishing");
|
||||
println!(
|
||||
" (review-queue persistence not yet implemented — this is where it plugs in)"
|
||||
);
|
||||
}
|
||||
(_, false) => {
|
||||
println!(" verification failed — not publishing, not updating state");
|
||||
}
|
||||
if let Err(err) = process_package(&client, &endpoints, state_dir, work_dir, name, pkg) {
|
||||
eprintln!(" error: {err:#}");
|
||||
any_failed = true;
|
||||
}
|
||||
}
|
||||
|
||||
if any_failed {
|
||||
bail!("one or more packages failed — see errors above");
|
||||
}
|
||||
Ok(())
|
||||
}
|
||||
|
||||
fn process_package(
|
||||
client: &reqwest::blocking::Client,
|
||||
endpoints: &GithubEndpoints,
|
||||
state_dir: &Path,
|
||||
work_dir: &Path,
|
||||
name: &str,
|
||||
pkg: &Package,
|
||||
) -> Result<()> {
|
||||
let latest = checker::latest_github_release(client, endpoints, &pkg.repo)?;
|
||||
let last_seen = state::load_last_version(state_dir, name);
|
||||
if last_seen.as_deref() == Some(latest.as_str()) {
|
||||
println!(" up to date at {latest}");
|
||||
return Ok(());
|
||||
}
|
||||
println!(" new version detected: {latest} (previously: {last_seen:?})");
|
||||
|
||||
let fetched = fetch_and_verify(client, endpoints, work_dir, name, pkg, &latest)?;
|
||||
println!(" fetched {}", fetched.asset.path.display());
|
||||
println!(
|
||||
" verification (tier {}): {} — {}",
|
||||
fetched.verification.tier,
|
||||
if fetched.verification.passed {
|
||||
"PASS"
|
||||
} else {
|
||||
"FAIL"
|
||||
},
|
||||
fetched.verification.justification
|
||||
);
|
||||
|
||||
if !fetched.verification.passed {
|
||||
println!(" verification failed — not publishing, not updating state");
|
||||
return Ok(());
|
||||
}
|
||||
|
||||
if fetched.verification.tier <= 3 {
|
||||
println!(" tier 1-3 pass: building + publishing");
|
||||
build_and_publish(name, pkg, &latest, &fetched)?;
|
||||
state::save_last_version(state_dir, name, &latest)?;
|
||||
state::clear_pending_version(state_dir, name)?;
|
||||
println!(" published {name} {latest}");
|
||||
} else if state::load_pending_version(state_dir, name).as_deref() == Some(latest.as_str()) {
|
||||
println!(" tier 4-6 pass: still pending review (`pkgwatch review` to see it)");
|
||||
} else {
|
||||
state::save_pending_version(state_dir, name, &latest)?;
|
||||
println!(" tier 4-6 pass: flagged for human review (`pkgwatch review` to approve)");
|
||||
}
|
||||
Ok(())
|
||||
}
|
||||
|
||||
struct FetchVerifyResult {
|
||||
version: String,
|
||||
asset_name: String,
|
||||
dest_dir: PathBuf,
|
||||
asset: DownloadedAsset,
|
||||
verification: VerificationResult,
|
||||
}
|
||||
|
||||
/// Shared by the normal check loop (tier 1-3 auto-path) and `pkgwatch
|
||||
/// review --approve` (which re-verifies before publishing rather than
|
||||
/// trusting a possibly-stale flag from an earlier run).
|
||||
fn fetch_and_verify(
|
||||
client: &reqwest::blocking::Client,
|
||||
endpoints: &GithubEndpoints,
|
||||
work_dir: &Path,
|
||||
name: &str,
|
||||
pkg: &Package,
|
||||
tag: &str,
|
||||
) -> Result<FetchVerifyResult> {
|
||||
let version = checker::version_from_tag(tag).to_string();
|
||||
let asset_name = pkg.asset_pattern.replace("{version}", &version);
|
||||
let dest_dir = work_dir.join(name).join(tag);
|
||||
|
||||
let asset = fetcher::download_asset(client, endpoints, &pkg.repo, tag, &asset_name, &dest_dir)?;
|
||||
let verification = verifier::verify(
|
||||
client,
|
||||
endpoints,
|
||||
&pkg.verification,
|
||||
&pkg.repo,
|
||||
tag,
|
||||
&asset.path,
|
||||
&dest_dir,
|
||||
)?;
|
||||
|
||||
Ok(FetchVerifyResult {
|
||||
version,
|
||||
asset_name,
|
||||
dest_dir,
|
||||
asset,
|
||||
verification,
|
||||
})
|
||||
}
|
||||
|
||||
fn build_and_publish(
|
||||
name: &str,
|
||||
pkg: &Package,
|
||||
tag: &str,
|
||||
fetched: &FetchVerifyResult,
|
||||
) -> Result<()> {
|
||||
let build_dir = fetched.dest_dir.join("build");
|
||||
let req = builder::BuildRequest {
|
||||
pkg_name: name,
|
||||
pkg,
|
||||
version: &fetched.version,
|
||||
repo: &pkg.repo,
|
||||
asset_name: &fetched.asset_name,
|
||||
download_url: &fetched.asset.download_url,
|
||||
artifact_path: &fetched.asset.path,
|
||||
};
|
||||
let built = builder::build(&req, &build_dir)?;
|
||||
println!(" built {}", built.package_path.display());
|
||||
|
||||
if let Some(check) = &pkg.sanity_check {
|
||||
let bin_dir = built.pkgdir.join("usr/bin");
|
||||
sanity::run(check, &bin_dir, &fetched.version)
|
||||
.with_context(|| format!("sanity check for {name} {tag}"))?;
|
||||
println!(" sanity check passed");
|
||||
}
|
||||
|
||||
let repo_dir = custom_repo_dir()?;
|
||||
let published = publisher::publish(&built.package_path, &repo_dir, CUSTOM_REPO_NAME)?;
|
||||
println!(
|
||||
" added to {} repo: {}",
|
||||
CUSTOM_REPO_NAME,
|
||||
published.display()
|
||||
);
|
||||
println!(
|
||||
" not installed automatically — run `sudo pacman -Syu` (or `sudo pacman -S {name}`) to pick it up"
|
||||
);
|
||||
Ok(())
|
||||
}
|
||||
|
||||
fn run_review(args: &[String]) -> Result<()> {
|
||||
let packages_dir = Path::new(PACKAGES_DIR);
|
||||
let state_dir = Path::new(STATE_DIR);
|
||||
let work_dir = Path::new(WORK_DIR);
|
||||
let packages = config::load_packages_dir(packages_dir)?;
|
||||
|
||||
match args {
|
||||
[] => {
|
||||
let mut any = false;
|
||||
for (name, _) in &packages {
|
||||
if let Some(pending) = state::load_pending_version(state_dir, name) {
|
||||
println!(
|
||||
"{name}: {pending} pending review (run `pkgwatch review {name} --approve`)"
|
||||
);
|
||||
any = true;
|
||||
}
|
||||
}
|
||||
if !any {
|
||||
println!("no packages pending review");
|
||||
}
|
||||
Ok(())
|
||||
}
|
||||
[name, flag] if flag == "--approve" => {
|
||||
let (_, pkg) = packages.iter().find(|(n, _)| n == name).with_context(|| {
|
||||
format!("no package named '{name}' in {}/", packages_dir.display())
|
||||
})?;
|
||||
let tag = state::load_pending_version(state_dir, name)
|
||||
.with_context(|| format!("'{name}' has no pending review"))?;
|
||||
approve(state_dir, work_dir, name, pkg, &tag)
|
||||
}
|
||||
_ => bail!("usage: pkgwatch review [<name> --approve]"),
|
||||
}
|
||||
}
|
||||
|
||||
fn approve(state_dir: &Path, work_dir: &Path, name: &str, pkg: &Package, tag: &str) -> Result<()> {
|
||||
let client = build_client()?;
|
||||
let endpoints = GithubEndpoints::default();
|
||||
|
||||
// Re-verify rather than trusting the earlier flag: the artifact at
|
||||
// this tag could in principle have changed since it was queued.
|
||||
let fetched = fetch_and_verify(&client, &endpoints, work_dir, name, pkg, tag)?;
|
||||
if !fetched.verification.passed {
|
||||
bail!(
|
||||
"re-verification failed on approve: {}",
|
||||
fetched.verification.justification
|
||||
);
|
||||
}
|
||||
println!(
|
||||
"re-verified (tier {}): {}",
|
||||
fetched.verification.tier, fetched.verification.justification
|
||||
);
|
||||
|
||||
build_and_publish(name, pkg, tag, &fetched)?;
|
||||
state::save_last_version(state_dir, name, tag)?;
|
||||
state::clear_pending_version(state_dir, name)?;
|
||||
println!("approved and published {name} {tag}");
|
||||
Ok(())
|
||||
}
|
||||
|
|
|
|||
123
src/publisher.rs
Normal file
123
src/publisher.rs
Normal file
|
|
@ -0,0 +1,123 @@
|
|||
use anyhow::{Context, Result, bail};
|
||||
use std::path::{Path, PathBuf};
|
||||
use std::process::Command;
|
||||
|
||||
/// Copies the built package into `repo_dir` and runs `repo-add` against
|
||||
/// `<repo_name>.db.tar.gz` there.
|
||||
///
|
||||
/// `repo_dir`/`repo_name` are expected to already be a real, registered
|
||||
/// pacman repo (see SPEC.md > Architecture > Publisher and
|
||||
/// `/etc/pacman.conf`'s `[custom]` section on this box) — pkgwatch doesn't
|
||||
/// create the repo or touch pacman.conf, only adds packages to an
|
||||
/// already-registered one. Getting the new version into an installed
|
||||
/// system is a separate, deliberate `pacman -Syu`/`pacman -S` step left to
|
||||
/// the operator, not run automatically here.
|
||||
pub fn publish(package_path: &Path, repo_dir: &Path, repo_name: &str) -> Result<PathBuf> {
|
||||
publish_with(Path::new("repo-add"), package_path, repo_dir, repo_name)
|
||||
}
|
||||
|
||||
/// `repo_add_bin` is injectable so tests can point it at a stub script
|
||||
/// instead of the real `repo-add` (or a mutated global `PATH`, which would
|
||||
/// race with `cargo test`'s parallel test threads).
|
||||
fn publish_with(
|
||||
repo_add_bin: &Path,
|
||||
package_path: &Path,
|
||||
repo_dir: &Path,
|
||||
repo_name: &str,
|
||||
) -> Result<PathBuf> {
|
||||
std::fs::create_dir_all(repo_dir)
|
||||
.with_context(|| format!("creating repo dir {}", repo_dir.display()))?;
|
||||
let file_name = package_path
|
||||
.file_name()
|
||||
.context("built package path has no filename")?;
|
||||
let dest = repo_dir.join(file_name);
|
||||
std::fs::copy(package_path, &dest)
|
||||
.with_context(|| format!("copying {} to {}", package_path.display(), dest.display()))?;
|
||||
|
||||
let db_path = repo_dir.join(format!("{repo_name}.db.tar.gz"));
|
||||
let status = Command::new(repo_add_bin)
|
||||
.arg(&db_path)
|
||||
.arg(&dest)
|
||||
.status()
|
||||
.with_context(|| {
|
||||
format!(
|
||||
"running {} (is pacman-contrib installed?)",
|
||||
repo_add_bin.display()
|
||||
)
|
||||
})?;
|
||||
if !status.success() {
|
||||
bail!("repo-add failed for {}", dest.display());
|
||||
}
|
||||
Ok(dest)
|
||||
}
|
||||
|
||||
#[cfg(test)]
|
||||
mod tests {
|
||||
use super::*;
|
||||
|
||||
fn write_stub(dir: &Path, name: &str, script: &str) -> PathBuf {
|
||||
let path = dir.join(name);
|
||||
std::fs::write(&path, format!("#!/bin/sh\n{script}\n")).unwrap();
|
||||
let mut perms = std::fs::metadata(&path).unwrap().permissions();
|
||||
std::os::unix::fs::PermissionsExt::set_mode(&mut perms, 0o755);
|
||||
std::fs::set_permissions(&path, perms).unwrap();
|
||||
path
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn publish_copies_package_and_invokes_repo_add() {
|
||||
let stub_dir = tempfile::tempdir().unwrap();
|
||||
let log_path = stub_dir.path().join("invoked_with.txt");
|
||||
let repo_add = write_stub(
|
||||
stub_dir.path(),
|
||||
"fake-repo-add",
|
||||
&format!("echo \"$@\" > {}", log_path.display()),
|
||||
);
|
||||
|
||||
let src_dir = tempfile::tempdir().unwrap();
|
||||
let package_path = src_dir.path().join("thing-1.0.0-1-x86_64.pkg.tar.zst");
|
||||
std::fs::write(&package_path, b"pkg-bytes").unwrap();
|
||||
|
||||
let repo_dir = tempfile::tempdir().unwrap();
|
||||
let dest = publish_with(&repo_add, &package_path, repo_dir.path(), "custom").unwrap();
|
||||
|
||||
assert_eq!(
|
||||
dest,
|
||||
repo_dir.path().join("thing-1.0.0-1-x86_64.pkg.tar.zst")
|
||||
);
|
||||
assert_eq!(std::fs::read(&dest).unwrap(), b"pkg-bytes");
|
||||
|
||||
let invoked_with = std::fs::read_to_string(&log_path).unwrap();
|
||||
assert!(invoked_with.contains("custom.db.tar.gz"));
|
||||
assert!(invoked_with.contains("thing-1.0.0-1-x86_64.pkg.tar.zst"));
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn publish_errors_when_repo_add_fails() {
|
||||
let stub_dir = tempfile::tempdir().unwrap();
|
||||
let repo_add = write_stub(stub_dir.path(), "fake-repo-add-fail", "exit 1");
|
||||
|
||||
let src_dir = tempfile::tempdir().unwrap();
|
||||
let package_path = src_dir.path().join("thing-1.0.0-1-x86_64.pkg.tar.zst");
|
||||
std::fs::write(&package_path, b"pkg-bytes").unwrap();
|
||||
|
||||
let repo_dir = tempfile::tempdir().unwrap();
|
||||
let err = publish_with(&repo_add, &package_path, repo_dir.path(), "custom").unwrap_err();
|
||||
assert!(err.to_string().contains("repo-add failed"));
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn publish_creates_repo_dir_if_missing() {
|
||||
let stub_dir = tempfile::tempdir().unwrap();
|
||||
let repo_add = write_stub(stub_dir.path(), "fake-repo-add-ok", "exit 0");
|
||||
|
||||
let src_dir = tempfile::tempdir().unwrap();
|
||||
let package_path = src_dir.path().join("thing-1.0.0-1-x86_64.pkg.tar.zst");
|
||||
std::fs::write(&package_path, b"pkg-bytes").unwrap();
|
||||
|
||||
let parent = tempfile::tempdir().unwrap();
|
||||
let repo_dir = parent.path().join("nested/repo");
|
||||
publish_with(&repo_add, &package_path, &repo_dir, "custom").unwrap();
|
||||
assert!(repo_dir.join("thing-1.0.0-1-x86_64.pkg.tar.zst").exists());
|
||||
}
|
||||
}
|
||||
125
src/sanity.rs
Normal file
125
src/sanity.rs
Normal file
|
|
@ -0,0 +1,125 @@
|
|||
use crate::config::SanityCheck;
|
||||
use anyhow::{Context, Result, bail};
|
||||
use regex::Regex;
|
||||
use std::path::Path;
|
||||
use std::process::Command;
|
||||
|
||||
/// Runs `check.command` with `pkg_bin_dir` prepended to `PATH`, so it
|
||||
/// exercises the binary pkgwatch just built (still sitting in makepkg's
|
||||
/// package staging directory, not installed system-wide) rather than
|
||||
/// whatever's already on the system. Confirms `check.version_regex`'s
|
||||
/// capture group matches `expected_version`.
|
||||
///
|
||||
/// Correctness check only, not a security control — see SPEC.md >
|
||||
/// Verification trust tiers. Catches checker bugs and mangled/wrong-asset
|
||||
/// downloads, not malicious releases.
|
||||
pub fn run(check: &SanityCheck, pkg_bin_dir: &Path, expected_version: &str) -> Result<()> {
|
||||
let path_env = format!(
|
||||
"{}:{}",
|
||||
pkg_bin_dir.display(),
|
||||
std::env::var("PATH").unwrap_or_default()
|
||||
);
|
||||
let output = Command::new("sh")
|
||||
.arg("-c")
|
||||
.arg(&check.command)
|
||||
.env("PATH", path_env)
|
||||
.output()
|
||||
.with_context(|| format!("running sanity check command '{}'", check.command))?;
|
||||
if !output.status.success() {
|
||||
bail!(
|
||||
"sanity check command '{}' exited with {}: {}",
|
||||
check.command,
|
||||
output.status,
|
||||
String::from_utf8_lossy(&output.stderr).trim()
|
||||
);
|
||||
}
|
||||
|
||||
let combined = format!(
|
||||
"{}{}",
|
||||
String::from_utf8_lossy(&output.stdout),
|
||||
String::from_utf8_lossy(&output.stderr)
|
||||
);
|
||||
let re = Regex::new(&check.version_regex)
|
||||
.with_context(|| format!("invalid version_regex '{}'", check.version_regex))?;
|
||||
let found = re
|
||||
.captures(&combined)
|
||||
.and_then(|caps| caps.get(1))
|
||||
.with_context(|| {
|
||||
format!(
|
||||
"version_regex '{}' did not match sanity check output: {combined:?}",
|
||||
check.version_regex
|
||||
)
|
||||
})?
|
||||
.as_str();
|
||||
|
||||
if found != expected_version {
|
||||
bail!(
|
||||
"sanity check reported version '{found}', pkgwatch built '{expected_version}' — mismatch"
|
||||
);
|
||||
}
|
||||
Ok(())
|
||||
}
|
||||
|
||||
#[cfg(test)]
|
||||
mod tests {
|
||||
use super::*;
|
||||
|
||||
fn write_fake_binary(dir: &Path, name: &str, script: &str) {
|
||||
let path = dir.join(name);
|
||||
std::fs::write(&path, format!("#!/bin/sh\n{script}\n")).unwrap();
|
||||
let mut perms = std::fs::metadata(&path).unwrap().permissions();
|
||||
std::os::unix::fs::PermissionsExt::set_mode(&mut perms, 0o755);
|
||||
std::fs::set_permissions(&path, perms).unwrap();
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn run_passes_when_reported_version_matches() {
|
||||
let dir = tempfile::tempdir().unwrap();
|
||||
write_fake_binary(dir.path(), "uv", "echo 'uv 0.12.15 (abc 2026-09-01)'");
|
||||
|
||||
let check = SanityCheck {
|
||||
command: "uv --version".to_string(),
|
||||
version_regex: r"uv (\d+\.\d+\.\d+)".to_string(),
|
||||
};
|
||||
assert!(run(&check, dir.path(), "0.12.15").is_ok());
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn run_fails_when_reported_version_differs() {
|
||||
let dir = tempfile::tempdir().unwrap();
|
||||
write_fake_binary(dir.path(), "uv", "echo 'uv 0.12.14 (abc 2026-08-01)'");
|
||||
|
||||
let check = SanityCheck {
|
||||
command: "uv --version".to_string(),
|
||||
version_regex: r"uv (\d+\.\d+\.\d+)".to_string(),
|
||||
};
|
||||
let err = run(&check, dir.path(), "0.12.15").unwrap_err();
|
||||
assert!(err.to_string().contains("mismatch"));
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn run_fails_when_command_exits_nonzero() {
|
||||
let dir = tempfile::tempdir().unwrap();
|
||||
write_fake_binary(dir.path(), "uv", "exit 1");
|
||||
|
||||
let check = SanityCheck {
|
||||
command: "uv --version".to_string(),
|
||||
version_regex: r"uv (\d+\.\d+\.\d+)".to_string(),
|
||||
};
|
||||
let err = run(&check, dir.path(), "0.12.15").unwrap_err();
|
||||
assert!(err.to_string().contains("exited with"));
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn run_fails_when_output_does_not_match_regex() {
|
||||
let dir = tempfile::tempdir().unwrap();
|
||||
write_fake_binary(dir.path(), "uv", "echo 'not a version'");
|
||||
|
||||
let check = SanityCheck {
|
||||
command: "uv --version".to_string(),
|
||||
version_regex: r"uv (\d+\.\d+\.\d+)".to_string(),
|
||||
};
|
||||
let err = run(&check, dir.path(), "0.12.15").unwrap_err();
|
||||
assert!(err.to_string().contains("did not match"));
|
||||
}
|
||||
}
|
||||
72
src/state.rs
72
src/state.rs
|
|
@ -17,6 +17,33 @@ pub fn save_last_version(state_dir: &Path, name: &str, version: &str) -> Result<
|
|||
Ok(())
|
||||
}
|
||||
|
||||
/// Tag currently awaiting human review for a tier 4-6 package (see
|
||||
/// SPEC.md > Architecture > Reviewer queue), if any. Separate from
|
||||
/// `load_last_version`/`save_last_version`: approving a review doesn't
|
||||
/// mean future versions auto-publish, so the two must be tracked
|
||||
/// independently.
|
||||
pub fn load_pending_version(state_dir: &Path, name: &str) -> Option<String> {
|
||||
std::fs::read_to_string(state_dir.join(format!("{name}.pending")))
|
||||
.ok()
|
||||
.map(|s| s.trim().to_string())
|
||||
}
|
||||
|
||||
pub fn save_pending_version(state_dir: &Path, name: &str, version: &str) -> Result<()> {
|
||||
std::fs::create_dir_all(state_dir)?;
|
||||
std::fs::write(state_dir.join(format!("{name}.pending")), version)?;
|
||||
Ok(())
|
||||
}
|
||||
|
||||
/// Clears a pending review, e.g. once it's been approved and published.
|
||||
/// Not an error if there was nothing pending.
|
||||
pub fn clear_pending_version(state_dir: &Path, name: &str) -> Result<()> {
|
||||
match std::fs::remove_file(state_dir.join(format!("{name}.pending"))) {
|
||||
Ok(()) => Ok(()),
|
||||
Err(e) if e.kind() == std::io::ErrorKind::NotFound => Ok(()),
|
||||
Err(e) => Err(e.into()),
|
||||
}
|
||||
}
|
||||
|
||||
#[cfg(test)]
|
||||
mod tests {
|
||||
use super::*;
|
||||
|
|
@ -57,4 +84,49 @@ mod tests {
|
|||
Some("0.12.15".to_string())
|
||||
);
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn load_pending_version_missing_file_returns_none() {
|
||||
let dir = tempfile::tempdir().unwrap();
|
||||
assert_eq!(load_pending_version(dir.path(), "scaleway-cli"), None);
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn save_then_load_pending_roundtrips() {
|
||||
let dir = tempfile::tempdir().unwrap();
|
||||
save_pending_version(dir.path(), "scaleway-cli", "v2.62.0").unwrap();
|
||||
assert_eq!(
|
||||
load_pending_version(dir.path(), "scaleway-cli"),
|
||||
Some("v2.62.0".to_string())
|
||||
);
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn clear_pending_version_removes_it() {
|
||||
let dir = tempfile::tempdir().unwrap();
|
||||
save_pending_version(dir.path(), "scaleway-cli", "v2.62.0").unwrap();
|
||||
clear_pending_version(dir.path(), "scaleway-cli").unwrap();
|
||||
assert_eq!(load_pending_version(dir.path(), "scaleway-cli"), None);
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn clear_pending_version_is_a_noop_when_nothing_pending() {
|
||||
let dir = tempfile::tempdir().unwrap();
|
||||
assert!(clear_pending_version(dir.path(), "scaleway-cli").is_ok());
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn pending_and_last_version_are_tracked_independently() {
|
||||
let dir = tempfile::tempdir().unwrap();
|
||||
save_last_version(dir.path(), "scaleway-cli", "v2.61.0").unwrap();
|
||||
save_pending_version(dir.path(), "scaleway-cli", "v2.62.0").unwrap();
|
||||
assert_eq!(
|
||||
load_last_version(dir.path(), "scaleway-cli"),
|
||||
Some("v2.61.0".to_string())
|
||||
);
|
||||
assert_eq!(
|
||||
load_pending_version(dir.path(), "scaleway-cli"),
|
||||
Some("v2.62.0".to_string())
|
||||
);
|
||||
}
|
||||
}
|
||||
|
|
|
|||
|
|
@ -2,8 +2,8 @@ use crate::checker::version_from_tag;
|
|||
use crate::config::Verification;
|
||||
use crate::fetcher;
|
||||
use crate::github::GithubEndpoints;
|
||||
use crate::hash::sha256_hex;
|
||||
use anyhow::{Context, Result, bail};
|
||||
use sha2::{Digest, Sha256};
|
||||
use std::path::Path;
|
||||
use std::process::Command;
|
||||
|
||||
|
|
@ -31,7 +31,7 @@ pub fn verify(
|
|||
} => {
|
||||
let checksum_asset_name =
|
||||
checksum_asset_pattern.replace("{version}", version_from_tag(tag));
|
||||
let checksum_path = fetcher::download_asset(
|
||||
let checksum_asset = fetcher::download_asset(
|
||||
client,
|
||||
endpoints,
|
||||
repo,
|
||||
|
|
@ -39,7 +39,7 @@ pub fn verify(
|
|||
&checksum_asset_name,
|
||||
dest_dir,
|
||||
)?;
|
||||
let checksum_text = std::fs::read_to_string(&checksum_path)?;
|
||||
let checksum_text = std::fs::read_to_string(&checksum_asset.path)?;
|
||||
let artifact_name = artifact_path
|
||||
.file_name()
|
||||
.and_then(|n| n.to_str())
|
||||
|
|
@ -94,12 +94,6 @@ pub fn verify(
|
|||
}
|
||||
}
|
||||
|
||||
fn sha256_hex(data: &[u8]) -> String {
|
||||
let mut hasher = Sha256::new();
|
||||
hasher.update(data);
|
||||
hex::encode(hasher.finalize())
|
||||
}
|
||||
|
||||
/// Finds the expected hash for `artifact_name` in a checksum file.
|
||||
///
|
||||
/// Handles both a bare-hash file covering a single asset (e.g. uv's
|
||||
|
|
|
|||
Loading…
Reference in a new issue