Compare commits
No commits in common. "master" and "worktree-add-wrapper-env-support" have entirely different histories.
master
...
worktree-a
20 changed files with 306 additions and 1147 deletions
|
|
@ -35,10 +35,8 @@ practice rather than asserted from habit — see Further reading.
|
||||||
`helpers/`) tends toward the opposite, and a single feature change ends
|
`helpers/`) tends toward the opposite, and a single feature change ends
|
||||||
up touching files scattered across every layer.
|
up touching files scattered across every layer.
|
||||||
**Rule**: modules are named after what they do in the pipeline
|
**Rule**: modules are named after what they do in the pipeline
|
||||||
(`release_source`, `fetcher`, `verifier`, `builder`, `sanity`,
|
(`checker`, `fetcher`, `verifier`, `builder`, `sanity`, `publisher`,
|
||||||
`publisher`, `state`), not generic buckets. (`release_source` is a directory module: the
|
`state`), not generic buckets. A new pipeline stage gets a new module
|
||||||
`ReleaseSource` trait and one file per host, re-exported from its
|
|
||||||
`mod.rs` so the rest of the crate never names a host's file.) A new pipeline stage gets a new module
|
|
||||||
named after the stage, not a method bolted onto an existing one.
|
named after the stage, not a method bolted onto an existing one.
|
||||||
**Anti-example to keep watching for**: a `utils.rs` grab-bag. `hash.rs`
|
**Anti-example to keep watching for**: a `utils.rs` grab-bag. `hash.rs`
|
||||||
could look like one but isn't — it exists for exactly one piece of
|
could look like one but isn't — it exists for exactly one piece of
|
||||||
|
|
@ -63,9 +61,8 @@ practice rather than asserted from habit — see Further reading.
|
||||||
Same testability goal as #3, applied to the specific ways this program
|
Same testability goal as #3, applied to the specific ways this program
|
||||||
reaches outside itself. A consistent shape beats ad hoc mocking invented
|
reaches outside itself. A consistent shape beats ad hoc mocking invented
|
||||||
per call site.
|
per call site.
|
||||||
**Already in force**: `GithubEndpoints`/`ForgejoEndpoints` (the
|
**Already in force**: `GithubEndpoints` (checker/fetcher/verifier),
|
||||||
`ReleaseSource` implementations, whose API root is what fetcher/verifier
|
`repo_add_bin` and the pacman.conf path (publisher), `PKGWATCH_REPO_DIR`
|
||||||
take), `repo_add_bin` and the pacman.conf path (publisher), `PKGWATCH_REPO_DIR`
|
|
||||||
(main, for manual dry runs against a scratch repo instead of the real
|
(main, for manual dry runs against a scratch repo instead of the real
|
||||||
one). A new external call follows the same shape: production code calls
|
one). A new external call follows the same shape: production code calls
|
||||||
a thin wrapper with the real default; tests call the parameterized
|
a thin wrapper with the real default; tests call the parameterized
|
||||||
|
|
@ -109,8 +106,8 @@ practice rather than asserted from habit — see Further reading.
|
||||||
directory, but the same information — why this way and not the obvious
|
directory, but the same information — why this way and not the obvious
|
||||||
alternative — needs to live somewhere a future reader will actually see
|
alternative — needs to live somewhere a future reader will actually see
|
||||||
it: the doc comment on the thing itself.
|
it: the doc comment on the thing itself.
|
||||||
**Example already here**: `release_source/github.rs`'s doc comment on
|
**Example already here**: `checker.rs`'s doc comment on
|
||||||
`GithubEndpoints::latest_release` explains why the newest Atom-feed entry isn't
|
`latest_github_release` explains why the newest Atom-feed entry isn't
|
||||||
trusted outright (scaleway-cli's `-dbg1` tag has no real Release behind
|
trusted outright (scaleway-cli's `-dbg1` tag has no real Release behind
|
||||||
it) — the reasoning lives right next to the code it justifies, not in a
|
it) — the reasoning lives right next to the code it justifies, not in a
|
||||||
commit message or a separate design doc no one will find later.
|
commit message or a separate design doc no one will find later.
|
||||||
|
|
|
||||||
112
docs/SPEC.md
112
docs/SPEC.md
|
|
@ -287,33 +287,17 @@ implements `repo`, `asset_pattern`, and `verification.method`
|
||||||
matches the line by filename instead of assuming a single-hash file.
|
matches the line by filename instead of assuming a single-hash file.
|
||||||
- Separately, scaleway-cli's Atom feed lists a `vX.Y.Z-dbg1` tag newest,
|
- Separately, scaleway-cli's Atom feed lists a `vX.Y.Z-dbg1` tag newest,
|
||||||
with no real Release object behind it (`releases/tags/<tag>` 404s) —
|
with no real Release object behind it (`releases/tags/<tag>` 404s) —
|
||||||
the GitHub source's `latest_release` now confirms each feed candidate
|
`checker::latest_github_release` now confirms each feed candidate
|
||||||
against the releases API in feed order rather than trusting the first
|
against the releases API in feed order rather than trusting the first
|
||||||
entry outright.
|
entry outright.
|
||||||
|
|
||||||
`sanity_check` and `binary_name` are now real, implemented fields (see
|
`sanity_check` and `binary_name` are now real, implemented fields (see
|
||||||
Builder/Sanity checker above) — added `packages.d/uv.toml`'s and
|
Builder/Sanity checker above) — added `packages.d/uv.toml`'s and
|
||||||
`packages.d/scaleway-cli.toml`'s own `sanity_check` blocks, and
|
`packages.d/scaleway-cli.toml`'s own `sanity_check` blocks, and
|
||||||
scaleway-cli's `binary_name = "scw"`. `source` is implemented too, with
|
scaleway-cli's `binary_name = "scw"`. `source`, `check_method`, and
|
||||||
two values: `github-release` (the default when omitted, so existing
|
`check_interval` are still schema sketch, not yet read by the code — the
|
||||||
configs are unchanged) and `forgejo-release`, which also requires a
|
PoC only knows how to check GitHub-release sources, on a single one-shot
|
||||||
`base_url` (see Checker below). `check_method` and `check_interval` are
|
run rather than a scheduled loop.
|
||||||
still schema sketch, not yet read by the code — checks are a fixed hourly
|
|
||||||
tick, not per-package.
|
|
||||||
|
|
||||||
```toml
|
|
||||||
# A package released from a Forgejo instance instead of GitHub. Only
|
|
||||||
# `same-origin-sha256` is valid here: `github-attestation` needs GitHub.
|
|
||||||
[package.mytool]
|
|
||||||
source = "forgejo-release"
|
|
||||||
base_url = "https://code.austinschaefer.com"
|
|
||||||
repo = "schaefera/mytool"
|
|
||||||
asset_pattern = "mytool-linux-x86_64.tar.gz"
|
|
||||||
|
|
||||||
[package.mytool.verification]
|
|
||||||
method = "same-origin-sha256"
|
|
||||||
checksum_asset_pattern = "SHA256SUMS"
|
|
||||||
```
|
|
||||||
|
|
||||||
Build/publish/review-queue (`makepkg`, `repo-add`, tier 4–6 human review)
|
Build/publish/review-queue (`makepkg`, `repo-add`, tier 4–6 human review)
|
||||||
are now implemented too — see Builder/Sanity checker/Publisher/Reviewer
|
are now implemented too — see Builder/Sanity checker/Publisher/Reviewer
|
||||||
|
|
@ -346,62 +330,19 @@ Open questions on the schema:
|
||||||
|
|
||||||
- **Config loader**: parses `packages.d/*.toml` into an in-memory package
|
- **Config loader**: parses `packages.d/*.toml` into an in-memory package
|
||||||
list. *(Implemented — `src/config.rs`.)*
|
list. *(Implemented — `src/config.rs`.)*
|
||||||
- **Paths**: where config, state and work files live, resolved by
|
|
||||||
`src/paths.rs` per the XDG base-directory spec rather than the current
|
|
||||||
working directory, so an installed binary behaves the same wherever it's
|
|
||||||
launched from. *(Implemented.)*
|
|
||||||
|
|
||||||
| What | Default | XDG variable | Override |
|
|
||||||
|---|---|---|---|
|
|
||||||
| Package declarations (`packages.d/*.toml`) | `~/.config/pkgwatch/packages.d` | `XDG_CONFIG_HOME` | `PKGWATCH_CONFIG_DIR` (the dir *containing* `packages.d`) |
|
|
||||||
| Last-published / pending versions | `~/.local/state/pkgwatch` | `XDG_STATE_HOME` | `PKGWATCH_STATE_DIR` |
|
|
||||||
| Downloads and build trees (safe to delete) | `~/.cache/pkgwatch` | `XDG_CACHE_HOME` | `PKGWATCH_WORK_DIR` |
|
|
||||||
|
|
||||||
Precedence per directory: override, then the XDG variable, then the
|
|
||||||
default under `$HOME`; an empty variable counts as unset. The overrides
|
|
||||||
are used verbatim (no `pkgwatch/` suffix) and exist for dry runs against
|
|
||||||
scratch directories, like `PKGWATCH_REPO_DIR` does for the pacman repo.
|
|
||||||
The XDG variables and `$HOME` must be absolute paths: a relative XDG
|
|
||||||
value is ignored, as the XDG spec requires, and a relative `$HOME` is an
|
|
||||||
error.
|
|
||||||
|
|
||||||
The checkout's `packages.d/` is no longer read on its own; it's just the
|
|
||||||
source to link from. Migrating from the old cwd-relative layout: move
|
|
||||||
`state/` to the state dir and copy or symlink `packages.d/` into the
|
|
||||||
config dir; `work/` is cache and can simply be dropped.
|
|
||||||
- **Checker**: per source type, resolves "what's the latest version" —
|
- **Checker**: per source type, resolves "what's the latest version" —
|
||||||
likely reuses `nvchecker`'s logic/sources conceptually for non-GitHub
|
likely reuses `nvchecker`'s logic/sources conceptually for non-GitHub
|
||||||
sources eventually. For GitHub sources, prefers the `github-atom` feed
|
sources eventually. For GitHub sources, prefers the `github-atom` feed
|
||||||
(see Scaling > Check method) over unconditional REST polling.
|
(see Scaling > Check method) over unconditional REST polling.
|
||||||
*(Implemented for GitHub and Forgejo. `src/release_source/` defines the
|
*(Implemented for GitHub only — `src/checker.rs` regex-matches the first
|
||||||
`ReleaseSource` trait (latest release + API root); each host implements
|
`releases/tag/<tag>` link in the feed rather than doing a full XML parse;
|
||||||
it in its own file (`github.rs`, `forgejo.rs`), and the module's
|
fine while the feed's newest-entry-first shape holds, revisit if that
|
||||||
`for_package` picks one per package, so adding a host doesn't touch
|
ever changes. `check_interval`/per-package cadence not wired up yet —
|
||||||
existing ones. The rest of the crate imports the trait and hosts from
|
the PoC is a single one-shot run, not a scheduled loop.)*
|
||||||
`crate::release_source`, which re-exports them. A trait
|
|
||||||
rather than an enum match because there are now two real hosts with
|
|
||||||
genuinely different logic. GitHub regex-matches the first
|
|
||||||
`releases/tag/<tag>` link in the feed rather than doing a full XML
|
|
||||||
parse; fine while the feed's newest-entry-first shape holds, revisit if
|
|
||||||
that ever changes. Forgejo is one call to
|
|
||||||
`<base_url>/api/v1/repos/<repo>/releases/latest`, which already returns
|
|
||||||
only the newest non-draft, non-prerelease release, so it needs none of
|
|
||||||
GitHub's confirm-each-tag step. `check_interval`/per-package cadence not
|
|
||||||
wired up yet — checks are a fixed hourly tick.)*
|
|
||||||
|
|
||||||
The HTTP is hand-rolled on the `reqwest` already in the tree, not an
|
|
||||||
API-client crate: pkgwatch needs two `GET`s, GitHub's check deliberately
|
|
||||||
uses an Atom feed no API crate covers (to stay off the rate-limited REST
|
|
||||||
API), and the release-by-tag call is shared verbatim by both hosts.
|
|
||||||
`octocrab` is async against our blocking `reqwest` with a default tree
|
|
||||||
larger than pkgwatch's whole current one; `forgejo-api` is a generated
|
|
||||||
binding of the entire API for one endpoint. Revisit if pkgwatch needs
|
|
||||||
authenticated or write API calls.
|
|
||||||
- **Fetcher**: downloads the artifact (and any checksum/signature/
|
- **Fetcher**: downloads the artifact (and any checksum/signature/
|
||||||
attestation companion) for a resolved version. *(Implemented —
|
attestation companion) for a resolved version. *(Implemented —
|
||||||
`src/fetcher.rs`, via the GitHub or Forgejo releases API — same
|
`src/fetcher.rs`, via the GitHub releases API; exact asset-name match,
|
||||||
`releases/tags/<tag>` endpoint and JSON shape on both; exact asset-name
|
not a glob.)*
|
||||||
match, not a glob.)*
|
|
||||||
- **Verifier**: tier-specific verification implementations, dispatched via
|
- **Verifier**: tier-specific verification implementations, dispatched via
|
||||||
a `Verification` enum matched on `method` (an internally-tagged serde
|
a `Verification` enum matched on `method` (an internally-tagged serde
|
||||||
enum) rather than a trait — simpler while there are only two methods;
|
enum) rather than a trait — simpler while there are only two methods;
|
||||||
|
|
@ -468,27 +409,7 @@ Open questions on the schema:
|
||||||
earlier run. No reject/dismiss command yet — see Status below.)*
|
earlier run. No reject/dismiss command yet — see Status below.)*
|
||||||
- **Scheduling**: systemd `.service` (oneshot) + `.timer` running it
|
- **Scheduling**: systemd `.service` (oneshot) + `.timer` running it
|
||||||
periodically, matching the pattern already used for other periodic tasks
|
periodically, matching the pattern already used for other periodic tasks
|
||||||
on this box. *(Implemented — user-level units under `systemd/`, run 10s
|
on this box. *(Not implemented — still a single one-shot `cargo run`.)*
|
||||||
after login and then hourly, non-persistent. Install from the main
|
|
||||||
checkout:*
|
|
||||||
|
|
||||||
```sh
|
|
||||||
cargo build --release && mkdir -p ~/.config/systemd/user &&
|
|
||||||
cp systemd/* ~/.config/systemd/user/ && systemctl --user daemon-reload &&
|
|
||||||
systemctl --user enable --now pkgwatch.timer
|
|
||||||
```
|
|
||||||
|
|
||||||
*`pkgwatch.service` runs the release binary from the checkout and sets
|
|
||||||
no `WorkingDirectory`: config, state and work dirs come from the XDG
|
|
||||||
paths above, so the service needs `~/.config/pkgwatch/packages.d` set
|
|
||||||
up first — see the migration note under Paths.)*
|
|
||||||
- **Notifications**: `notifier.rs` sends a desktop notification
|
|
||||||
(`notify-send`) when a tier 4-6 release is newly queued for review or a
|
|
||||||
tier 1-3 release is published; both are best-effort and never fail a
|
|
||||||
run. A non-zero exit (verification/build/network failure) triggers
|
|
||||||
`pkgwatch-failure.service` via `OnFailure=`. Approving via
|
|
||||||
`pkgwatch review --approve` doesn't notify — the operator is already at
|
|
||||||
the terminal.
|
|
||||||
|
|
||||||
## Prior art / reference points
|
## Prior art / reference points
|
||||||
|
|
||||||
|
|
@ -526,8 +447,8 @@ Open questions on the schema:
|
||||||
confirmed correct — no build-provenance attestations upstream.
|
confirmed correct — no build-provenance attestations upstream.
|
||||||
Required adding `{version}`-placeholder support to `asset_pattern`/
|
Required adding `{version}`-placeholder support to `asset_pattern`/
|
||||||
`checksum_asset_pattern`, filename-matched parsing of combined
|
`checksum_asset_pattern`, filename-matched parsing of combined
|
||||||
multi-asset checksum files, and having the GitHub source's
|
multi-asset checksum files, and having `latest_github_release`
|
||||||
`latest_release` confirm each Atom-feed candidate against the releases API (this
|
confirm each Atom-feed candidate against the releases API (this
|
||||||
repo's newest feed entry, a `-dbg1` tag, has no real Release behind
|
repo's newest feed entry, a `-dbg1` tag, has no real Release behind
|
||||||
it). Still just flags for human review, same as any tier 4-6 pass —
|
it). Still just flags for human review, same as any tier 4-6 pass —
|
||||||
not auto-installed; see the unchecked build/publish item below.
|
not auto-installed; see the unchecked build/publish item below.
|
||||||
|
|
@ -552,8 +473,7 @@ Open questions on the schema:
|
||||||
ever actually fails on it.
|
ever actually fails on it.
|
||||||
- [ ] Not yet implemented: `pkgwatch review <name> --reject` (a pending
|
- [ ] Not yet implemented: `pkgwatch review <name> --reject` (a pending
|
||||||
review can only be approved or left pending, not dismissed),
|
review can only be approved or left pending, not dismissed),
|
||||||
per-package `check_interval` (the timer is a fixed hourly tick),
|
scheduling/`check_interval`, non-GitHub sources, `minisign`/tier-1
|
||||||
sources other than GitHub and Forgejo releases, `minisign`/tier-1
|
|
||||||
method, retention/pruning of old versions in the local repo (see
|
method, retention/pruning of old versions in the local repo (see
|
||||||
Scaling > Local repo retention), staggering/auth for GitHub API
|
Scaling > Local repo retention), staggering/auth for GitHub API
|
||||||
rate limits at higher package counts.
|
rate limits at higher package counts.
|
||||||
|
|
|
||||||
125
src/checker.rs
125
src/checker.rs
|
|
@ -1,6 +1,45 @@
|
||||||
//! Turns a release tag into a version string. What the latest tag *is*
|
use crate::github::GithubEndpoints;
|
||||||
//! comes from a `ReleaseSource` (see `release_source`); this is the one piece of
|
use anyhow::{Result, bail};
|
||||||
//! the check stage that isn't host-specific.
|
use regex::Regex;
|
||||||
|
|
||||||
|
/// Resolves the latest release tag for `repo` via its public Atom feed.
|
||||||
|
///
|
||||||
|
/// Deliberately not a full XML parse: the feed lists entries newest-first,
|
||||||
|
/// and each `<link rel="alternate" .../releases/tag/<tag>"/>` is matched
|
||||||
|
/// in document order. Revisit with a real XML parser if GitHub's feed
|
||||||
|
/// shape ever changes.
|
||||||
|
///
|
||||||
|
/// The feed can list a tag newer than any tag with a real Release object
|
||||||
|
/// behind it — observed on scaleway/scaleway-cli, which pushes a
|
||||||
|
/// `vX.Y.Z-dbg1` tag (no corresponding Release; `releases/tags/<tag>`
|
||||||
|
/// 404s) right after each real release, and that tag sorts newest in the
|
||||||
|
/// feed. So each candidate is confirmed against the releases API in feed
|
||||||
|
/// order, returning the first that actually resolves.
|
||||||
|
pub fn latest_github_release(
|
||||||
|
client: &reqwest::blocking::Client,
|
||||||
|
endpoints: &GithubEndpoints,
|
||||||
|
repo: &str,
|
||||||
|
) -> Result<String> {
|
||||||
|
let url = format!("{}/{repo}/releases.atom", endpoints.web);
|
||||||
|
let body = client.get(&url).send()?.error_for_status()?.text()?;
|
||||||
|
|
||||||
|
let re = Regex::new(r#"releases/tag/([^"]+)""#)?;
|
||||||
|
let mut candidates = re
|
||||||
|
.captures_iter(&body)
|
||||||
|
.map(|caps| caps[1].to_string())
|
||||||
|
.peekable();
|
||||||
|
if candidates.peek().is_none() {
|
||||||
|
bail!("no release tag found in {url}");
|
||||||
|
}
|
||||||
|
|
||||||
|
for tag in candidates {
|
||||||
|
let release_url = format!("{}/repos/{repo}/releases/tags/{tag}", endpoints.api);
|
||||||
|
if client.get(&release_url).send()?.status().is_success() {
|
||||||
|
return Ok(tag);
|
||||||
|
}
|
||||||
|
}
|
||||||
|
bail!("no release tag in {url} resolved to a real release via the API")
|
||||||
|
}
|
||||||
|
|
||||||
/// Strips a leading `v` from a release tag, e.g. `v2.62.0` -> `2.62.0`.
|
/// Strips a leading `v` from a release tag, e.g. `v2.62.0` -> `2.62.0`.
|
||||||
///
|
///
|
||||||
|
|
@ -26,4 +65,84 @@ mod tests {
|
||||||
fn version_from_tag_leaves_bare_version_unchanged() {
|
fn version_from_tag_leaves_bare_version_unchanged() {
|
||||||
assert_eq!(version_from_tag("0.12.15"), "0.12.15");
|
assert_eq!(version_from_tag("0.12.15"), "0.12.15");
|
||||||
}
|
}
|
||||||
|
|
||||||
|
fn atom_feed(tags: &[&str]) -> String {
|
||||||
|
let entries: String = tags
|
||||||
|
.iter()
|
||||||
|
.map(|t| {
|
||||||
|
format!(r#"<link rel="alternate" href="https://github.com/o/r/releases/tag/{t}"/>"#)
|
||||||
|
})
|
||||||
|
.collect();
|
||||||
|
format!("<feed>{entries}</feed>")
|
||||||
|
}
|
||||||
|
|
||||||
|
#[test]
|
||||||
|
fn latest_github_release_skips_tags_with_no_real_release() {
|
||||||
|
let mut server = mockito::Server::new();
|
||||||
|
let endpoints = GithubEndpoints {
|
||||||
|
web: server.url(),
|
||||||
|
api: server.url(),
|
||||||
|
};
|
||||||
|
|
||||||
|
// Mirrors the real scaleway-cli case: newest feed entry (a -dbg1
|
||||||
|
// tag) has no Release object behind it and 404s.
|
||||||
|
let _feed = server
|
||||||
|
.mock("GET", "/o/r/releases.atom")
|
||||||
|
.with_status(200)
|
||||||
|
.with_body(atom_feed(&["v2.62.0-dbg1", "v2.62.0"]))
|
||||||
|
.create();
|
||||||
|
let _missing = server
|
||||||
|
.mock("GET", "/repos/o/r/releases/tags/v2.62.0-dbg1")
|
||||||
|
.with_status(404)
|
||||||
|
.create();
|
||||||
|
let _real = server
|
||||||
|
.mock("GET", "/repos/o/r/releases/tags/v2.62.0")
|
||||||
|
.with_status(200)
|
||||||
|
.with_body("{}")
|
||||||
|
.create();
|
||||||
|
|
||||||
|
let client = reqwest::blocking::Client::new();
|
||||||
|
let tag = latest_github_release(&client, &endpoints, "o/r").unwrap();
|
||||||
|
assert_eq!(tag, "v2.62.0");
|
||||||
|
}
|
||||||
|
|
||||||
|
#[test]
|
||||||
|
fn latest_github_release_errors_when_feed_has_no_tags() {
|
||||||
|
let mut server = mockito::Server::new();
|
||||||
|
let endpoints = GithubEndpoints {
|
||||||
|
web: server.url(),
|
||||||
|
api: server.url(),
|
||||||
|
};
|
||||||
|
let _feed = server
|
||||||
|
.mock("GET", "/o/r/releases.atom")
|
||||||
|
.with_status(200)
|
||||||
|
.with_body("<feed></feed>")
|
||||||
|
.create();
|
||||||
|
|
||||||
|
let client = reqwest::blocking::Client::new();
|
||||||
|
let err = latest_github_release(&client, &endpoints, "o/r").unwrap_err();
|
||||||
|
assert!(err.to_string().contains("no release tag found"));
|
||||||
|
}
|
||||||
|
|
||||||
|
#[test]
|
||||||
|
fn latest_github_release_errors_when_no_candidate_resolves() {
|
||||||
|
let mut server = mockito::Server::new();
|
||||||
|
let endpoints = GithubEndpoints {
|
||||||
|
web: server.url(),
|
||||||
|
api: server.url(),
|
||||||
|
};
|
||||||
|
let _feed = server
|
||||||
|
.mock("GET", "/o/r/releases.atom")
|
||||||
|
.with_status(200)
|
||||||
|
.with_body(atom_feed(&["v1.0.0-dbg1"]))
|
||||||
|
.create();
|
||||||
|
let _missing = server
|
||||||
|
.mock("GET", "/repos/o/r/releases/tags/v1.0.0-dbg1")
|
||||||
|
.with_status(404)
|
||||||
|
.create();
|
||||||
|
|
||||||
|
let client = reqwest::blocking::Client::new();
|
||||||
|
let err = latest_github_release(&client, &endpoints, "o/r").unwrap_err();
|
||||||
|
assert!(err.to_string().contains("resolved to a real release"));
|
||||||
|
}
|
||||||
}
|
}
|
||||||
|
|
|
||||||
128
src/config.rs
128
src/config.rs
|
|
@ -2,7 +2,7 @@
|
||||||
//! The only module that knows the TOML shape — everything downstream
|
//! The only module that knows the TOML shape — everything downstream
|
||||||
//! works with `Package`/`Verification`/`SanityCheck`, never raw TOML.
|
//! works with `Package`/`Verification`/`SanityCheck`, never raw TOML.
|
||||||
|
|
||||||
use anyhow::{Context, Result, bail};
|
use anyhow::{Context, Result};
|
||||||
use serde::Deserialize;
|
use serde::Deserialize;
|
||||||
use std::collections::{BTreeMap, HashMap};
|
use std::collections::{BTreeMap, HashMap};
|
||||||
use std::path::Path;
|
use std::path::Path;
|
||||||
|
|
@ -12,29 +12,10 @@ struct PackageFile {
|
||||||
package: HashMap<String, Package>,
|
package: HashMap<String, Package>,
|
||||||
}
|
}
|
||||||
|
|
||||||
/// Where a package's releases are published: the `source` key from
|
|
||||||
/// docs/SPEC.md > Config schema. Omitted means GitHub, so every existing
|
|
||||||
/// `packages.d/*.toml` keeps working.
|
|
||||||
#[derive(Debug, Deserialize, Clone, Copy, Default, PartialEq, Eq)]
|
|
||||||
#[serde(rename_all = "kebab-case")]
|
|
||||||
pub enum Source {
|
|
||||||
#[default]
|
|
||||||
GithubRelease,
|
|
||||||
/// A Forgejo (or Gitea) instance; needs `base_url` too.
|
|
||||||
ForgejoRelease,
|
|
||||||
}
|
|
||||||
|
|
||||||
#[derive(Debug, Deserialize, Clone)]
|
#[derive(Debug, Deserialize, Clone)]
|
||||||
pub struct Package {
|
pub struct Package {
|
||||||
/// `owner/name` on whichever `source` hosts it.
|
|
||||||
pub repo: String,
|
pub repo: String,
|
||||||
#[serde(default)]
|
/// Exact GitHub release asset name (still not a glob — see
|
||||||
pub source: Source,
|
|
||||||
/// Web root of the Forgejo instance, e.g. `https://code.austinschaefer.com`
|
|
||||||
/// (the API lives under `/api/v1`). Required for, and only meaningful
|
|
||||||
/// with, `source = "forgejo-release"`.
|
|
||||||
pub base_url: Option<String>,
|
|
||||||
/// Exact release asset name (still not a glob — see
|
|
||||||
/// docs/SPEC.md > Architecture > Fetcher), optionally containing a
|
/// docs/SPEC.md > Architecture > Fetcher), optionally containing a
|
||||||
/// `{version}` placeholder for projects whose asset names embed the
|
/// `{version}` placeholder for projects whose asset names embed the
|
||||||
/// version (e.g. `scaleway-cli_{version}_linux_amd64`). Substituted via
|
/// version (e.g. `scaleway-cli_{version}_linux_amd64`). Substituted via
|
||||||
|
|
@ -79,32 +60,6 @@ pub struct Package {
|
||||||
}
|
}
|
||||||
|
|
||||||
impl Package {
|
impl Package {
|
||||||
/// Rejects combinations that can't work, once at load time rather than
|
|
||||||
/// as a confusing failure deep in a run (see docs/ARCHITECTURE.md >
|
|
||||||
/// "validate at the boundary, once").
|
|
||||||
fn validate(&self, name: &str) -> Result<()> {
|
|
||||||
match (self.source, &self.base_url) {
|
|
||||||
(Source::GithubRelease, None) => {}
|
|
||||||
(Source::GithubRelease, Some(_)) => {
|
|
||||||
// Silently ignoring it would hide a mistyped `source`.
|
|
||||||
bail!("{name}: base_url only applies to source = \"forgejo-release\"");
|
|
||||||
}
|
|
||||||
(Source::ForgejoRelease, None) => {
|
|
||||||
bail!("{name}: source = \"forgejo-release\" needs a base_url");
|
|
||||||
}
|
|
||||||
(Source::ForgejoRelease, Some(url)) => {
|
|
||||||
if !url.starts_with("https://") && !url.starts_with("http://") {
|
|
||||||
bail!("{name}: base_url '{url}' must start with http:// or https://");
|
|
||||||
}
|
|
||||||
// `gh attestation verify` only speaks GitHub's attestation API.
|
|
||||||
if matches!(self.verification, Verification::GithubAttestation) {
|
|
||||||
bail!("{name}: github-attestation verification needs a GitHub source");
|
|
||||||
}
|
|
||||||
}
|
|
||||||
}
|
|
||||||
Ok(())
|
|
||||||
}
|
|
||||||
|
|
||||||
/// The name of the executable inside the built package: `binary_name`
|
/// The name of the executable inside the built package: `binary_name`
|
||||||
/// if the package declares one, else `pkg_name` itself.
|
/// if the package declares one, else `pkg_name` itself.
|
||||||
pub fn binary_name<'a>(&'a self, pkg_name: &'a str) -> &'a str {
|
pub fn binary_name<'a>(&'a self, pkg_name: &'a str) -> &'a str {
|
||||||
|
|
@ -152,10 +107,6 @@ pub fn load_packages_dir(dir: &Path) -> Result<Vec<(String, Package)>> {
|
||||||
.with_context(|| format!("reading {}", path.display()))?;
|
.with_context(|| format!("reading {}", path.display()))?;
|
||||||
let file: PackageFile =
|
let file: PackageFile =
|
||||||
toml::from_str(&text).with_context(|| format!("parsing {}", path.display()))?;
|
toml::from_str(&text).with_context(|| format!("parsing {}", path.display()))?;
|
||||||
for (name, pkg) in &file.package {
|
|
||||||
pkg.validate(name)
|
|
||||||
.with_context(|| format!("in {}", path.display()))?;
|
|
||||||
}
|
|
||||||
out.extend(file.package);
|
out.extend(file.package);
|
||||||
}
|
}
|
||||||
Ok(out)
|
Ok(out)
|
||||||
|
|
@ -307,79 +258,4 @@ mod tests {
|
||||||
let dir = tempfile::tempdir().unwrap();
|
let dir = tempfile::tempdir().unwrap();
|
||||||
assert!(load_packages_dir(dir.path()).unwrap().is_empty());
|
assert!(load_packages_dir(dir.path()).unwrap().is_empty());
|
||||||
}
|
}
|
||||||
|
|
||||||
/// One `[package.p]` with the given extra top-level lines and
|
|
||||||
/// verification table, loaded through the real loader so validation
|
|
||||||
/// runs too.
|
|
||||||
fn load_one(extra: &str, verification: &str) -> Result<Package> {
|
|
||||||
let dir = tempfile::tempdir().unwrap();
|
|
||||||
write(
|
|
||||||
dir.path(),
|
|
||||||
"p.toml",
|
|
||||||
&format!(
|
|
||||||
"[package.p]\nrepo = \"o/r\"\nasset_pattern = \"x\"\n{extra}\n\
|
|
||||||
[package.p.verification]\n{verification}\n"
|
|
||||||
),
|
|
||||||
);
|
|
||||||
let mut loaded = load_packages_dir(dir.path())?;
|
|
||||||
Ok(loaded.remove(0).1)
|
|
||||||
}
|
|
||||||
|
|
||||||
const SAME_ORIGIN: &str = "method = \"same-origin-sha256\"\nchecksum_asset_pattern = \"SUMS\"";
|
|
||||||
const ATTESTATION: &str = "method = \"github-attestation\"";
|
|
||||||
|
|
||||||
const FORGEJO: &str = "source = \"forgejo-release\"\nbase_url = \"https://forge.example.com\"";
|
|
||||||
|
|
||||||
#[test]
|
|
||||||
fn source_defaults_to_github_release() {
|
|
||||||
let pkg = load_one("", ATTESTATION).unwrap();
|
|
||||||
assert_eq!(pkg.source, Source::GithubRelease);
|
|
||||||
assert_eq!(pkg.base_url, None);
|
|
||||||
}
|
|
||||||
|
|
||||||
#[test]
|
|
||||||
fn loads_explicit_github_release_source() {
|
|
||||||
let pkg = load_one("source = \"github-release\"", ATTESTATION).unwrap();
|
|
||||||
assert_eq!(pkg.source, Source::GithubRelease);
|
|
||||||
}
|
|
||||||
|
|
||||||
#[test]
|
|
||||||
fn loads_forgejo_release_source() {
|
|
||||||
let pkg = load_one(FORGEJO, SAME_ORIGIN).unwrap();
|
|
||||||
assert_eq!(pkg.source, Source::ForgejoRelease);
|
|
||||||
assert_eq!(pkg.base_url.as_deref(), Some("https://forge.example.com"));
|
|
||||||
}
|
|
||||||
|
|
||||||
#[test]
|
|
||||||
fn rejects_forgejo_release_without_base_url() {
|
|
||||||
let err = load_one("source = \"forgejo-release\"", SAME_ORIGIN).unwrap_err();
|
|
||||||
assert!(format!("{err:#}").contains("needs a base_url"));
|
|
||||||
}
|
|
||||||
|
|
||||||
#[test]
|
|
||||||
fn rejects_base_url_on_a_github_source() {
|
|
||||||
let err = load_one("base_url = \"https://forge.example.com\"", SAME_ORIGIN).unwrap_err();
|
|
||||||
assert!(format!("{err:#}").contains("only applies to source"));
|
|
||||||
}
|
|
||||||
|
|
||||||
#[test]
|
|
||||||
fn rejects_forgejo_base_url_without_scheme() {
|
|
||||||
let err = load_one(
|
|
||||||
"source = \"forgejo-release\"\nbase_url = \"forge.example.com\"",
|
|
||||||
SAME_ORIGIN,
|
|
||||||
)
|
|
||||||
.unwrap_err();
|
|
||||||
assert!(format!("{err:#}").contains("must start with http"));
|
|
||||||
}
|
|
||||||
|
|
||||||
#[test]
|
|
||||||
fn rejects_github_attestation_on_a_forgejo_source() {
|
|
||||||
let err = load_one(FORGEJO, ATTESTATION).unwrap_err();
|
|
||||||
assert!(format!("{err:#}").contains("needs a GitHub source"));
|
|
||||||
}
|
|
||||||
|
|
||||||
#[test]
|
|
||||||
fn rejects_unknown_source() {
|
|
||||||
assert!(load_one("source = \"gitlab-release\"", SAME_ORIGIN).is_err());
|
|
||||||
}
|
|
||||||
}
|
}
|
||||||
|
|
|
||||||
|
|
@ -1,7 +1,8 @@
|
||||||
//! Downloads a named release asset (from GitHub or Forgejo) to a local
|
//! Downloads a named GitHub release asset to a local path. The only
|
||||||
//! path. The only module that talks to the releases API for asset bytes —
|
//! module that talks to the releases API for asset bytes — `checker` only
|
||||||
//! `release_source` only resolves version tags, never downloads.
|
//! resolves version tags, never downloads.
|
||||||
|
|
||||||
|
use crate::github::GithubEndpoints;
|
||||||
use anyhow::{Context, Result};
|
use anyhow::{Context, Result};
|
||||||
use serde::Deserialize;
|
use serde::Deserialize;
|
||||||
use std::path::{Path, PathBuf};
|
use std::path::{Path, PathBuf};
|
||||||
|
|
@ -28,18 +29,16 @@ pub struct DownloadedAsset {
|
||||||
}
|
}
|
||||||
|
|
||||||
/// Downloads the release asset named exactly `asset_name` for `repo`@`tag`
|
/// Downloads the release asset named exactly `asset_name` for `repo`@`tag`
|
||||||
/// into `dest_dir`, returning the local path and its origin URL. `api` is
|
/// into `dest_dir`, returning the local path and its origin URL.
|
||||||
/// the releases API root (see `ReleaseSource::api`); GitHub and Forgejo
|
|
||||||
/// serve the same endpoint and JSON shape under it.
|
|
||||||
pub fn download_asset(
|
pub fn download_asset(
|
||||||
client: &reqwest::blocking::Client,
|
client: &reqwest::blocking::Client,
|
||||||
api: &str,
|
endpoints: &GithubEndpoints,
|
||||||
repo: &str,
|
repo: &str,
|
||||||
tag: &str,
|
tag: &str,
|
||||||
asset_name: &str,
|
asset_name: &str,
|
||||||
dest_dir: &Path,
|
dest_dir: &Path,
|
||||||
) -> Result<DownloadedAsset> {
|
) -> Result<DownloadedAsset> {
|
||||||
let api_url = format!("{api}/repos/{repo}/releases/tags/{tag}");
|
let api_url = format!("{}/repos/{repo}/releases/tags/{tag}", endpoints.api);
|
||||||
let release: Release = client
|
let release: Release = client
|
||||||
.get(&api_url)
|
.get(&api_url)
|
||||||
.send()?
|
.send()?
|
||||||
|
|
@ -74,7 +73,10 @@ mod tests {
|
||||||
#[test]
|
#[test]
|
||||||
fn download_asset_writes_matching_asset_to_dest_dir() {
|
fn download_asset_writes_matching_asset_to_dest_dir() {
|
||||||
let mut server = mockito::Server::new();
|
let mut server = mockito::Server::new();
|
||||||
let api = server.url();
|
let endpoints = GithubEndpoints {
|
||||||
|
web: server.url(),
|
||||||
|
api: server.url(),
|
||||||
|
};
|
||||||
let asset_url = format!("{}/download/thing.tar.gz", server.url());
|
let asset_url = format!("{}/download/thing.tar.gz", server.url());
|
||||||
let release_body = format!(
|
let release_body = format!(
|
||||||
r#"{{"assets": [{{"name": "thing.tar.gz", "browser_download_url": "{asset_url}"}}]}}"#
|
r#"{{"assets": [{{"name": "thing.tar.gz", "browser_download_url": "{asset_url}"}}]}}"#
|
||||||
|
|
@ -94,7 +96,7 @@ mod tests {
|
||||||
let dest_dir = tempfile::tempdir().unwrap();
|
let dest_dir = tempfile::tempdir().unwrap();
|
||||||
let asset = download_asset(
|
let asset = download_asset(
|
||||||
&client,
|
&client,
|
||||||
&api,
|
&endpoints,
|
||||||
"o/r",
|
"o/r",
|
||||||
"v1.0.0",
|
"v1.0.0",
|
||||||
"thing.tar.gz",
|
"thing.tar.gz",
|
||||||
|
|
@ -110,7 +112,10 @@ mod tests {
|
||||||
#[test]
|
#[test]
|
||||||
fn download_asset_errors_when_no_asset_matches() {
|
fn download_asset_errors_when_no_asset_matches() {
|
||||||
let mut server = mockito::Server::new();
|
let mut server = mockito::Server::new();
|
||||||
let api = server.url();
|
let endpoints = GithubEndpoints {
|
||||||
|
web: server.url(),
|
||||||
|
api: server.url(),
|
||||||
|
};
|
||||||
let _release = server
|
let _release = server
|
||||||
.mock("GET", "/repos/o/r/releases/tags/v1.0.0")
|
.mock("GET", "/repos/o/r/releases/tags/v1.0.0")
|
||||||
.with_status(200)
|
.with_status(200)
|
||||||
|
|
@ -121,7 +126,7 @@ mod tests {
|
||||||
let dest_dir = tempfile::tempdir().unwrap();
|
let dest_dir = tempfile::tempdir().unwrap();
|
||||||
let err = download_asset(
|
let err = download_asset(
|
||||||
&client,
|
&client,
|
||||||
&api,
|
&endpoints,
|
||||||
"o/r",
|
"o/r",
|
||||||
"v1.0.0",
|
"v1.0.0",
|
||||||
"thing.tar.gz",
|
"thing.tar.gz",
|
||||||
|
|
@ -134,7 +139,10 @@ mod tests {
|
||||||
#[test]
|
#[test]
|
||||||
fn download_asset_errors_when_release_not_found() {
|
fn download_asset_errors_when_release_not_found() {
|
||||||
let mut server = mockito::Server::new();
|
let mut server = mockito::Server::new();
|
||||||
let api = server.url();
|
let endpoints = GithubEndpoints {
|
||||||
|
web: server.url(),
|
||||||
|
api: server.url(),
|
||||||
|
};
|
||||||
let _release = server
|
let _release = server
|
||||||
.mock("GET", "/repos/o/r/releases/tags/v1.0.0")
|
.mock("GET", "/repos/o/r/releases/tags/v1.0.0")
|
||||||
.with_status(404)
|
.with_status(404)
|
||||||
|
|
@ -144,7 +152,7 @@ mod tests {
|
||||||
let dest_dir = tempfile::tempdir().unwrap();
|
let dest_dir = tempfile::tempdir().unwrap();
|
||||||
let err = download_asset(
|
let err = download_asset(
|
||||||
&client,
|
&client,
|
||||||
&api,
|
&endpoints,
|
||||||
"o/r",
|
"o/r",
|
||||||
"v1.0.0",
|
"v1.0.0",
|
||||||
"thing.tar.gz",
|
"thing.tar.gz",
|
||||||
|
|
|
||||||
29
src/github.rs
Normal file
29
src/github.rs
Normal file
|
|
@ -0,0 +1,29 @@
|
||||||
|
/// Base URLs for GitHub's public web host (Atom feeds, release pages) and
|
||||||
|
/// its REST API, factored out so tests can point both at a local mock
|
||||||
|
/// server instead of the real github.com/api.github.com.
|
||||||
|
#[derive(Debug, Clone)]
|
||||||
|
pub struct GithubEndpoints {
|
||||||
|
pub web: String,
|
||||||
|
pub api: String,
|
||||||
|
}
|
||||||
|
|
||||||
|
impl Default for GithubEndpoints {
|
||||||
|
fn default() -> Self {
|
||||||
|
Self {
|
||||||
|
web: "https://github.com".to_string(),
|
||||||
|
api: "https://api.github.com".to_string(),
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
#[cfg(test)]
|
||||||
|
mod tests {
|
||||||
|
use super::*;
|
||||||
|
|
||||||
|
#[test]
|
||||||
|
fn default_points_at_real_github() {
|
||||||
|
let endpoints = GithubEndpoints::default();
|
||||||
|
assert_eq!(endpoints.web, "https://github.com");
|
||||||
|
assert_eq!(endpoints.api, "https://api.github.com");
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
@ -6,12 +6,10 @@ mod builder;
|
||||||
mod checker;
|
mod checker;
|
||||||
mod config;
|
mod config;
|
||||||
mod fetcher;
|
mod fetcher;
|
||||||
|
mod github;
|
||||||
mod hash;
|
mod hash;
|
||||||
mod notifier;
|
|
||||||
mod paths;
|
|
||||||
mod pipeline;
|
mod pipeline;
|
||||||
mod publisher;
|
mod publisher;
|
||||||
mod release_source;
|
|
||||||
mod sanity;
|
mod sanity;
|
||||||
mod state;
|
mod state;
|
||||||
#[cfg(test)]
|
#[cfg(test)]
|
||||||
|
|
|
||||||
111
src/notifier.rs
111
src/notifier.rs
|
|
@ -1,111 +0,0 @@
|
||||||
//! Tells the operator, via a desktop notification, that a package needs
|
|
||||||
//! attention (a tier 4-6 release awaiting review) or just landed in the
|
|
||||||
//! local repo. Best-effort: a missing `notify-send` or session bus must
|
|
||||||
//! never fail a run, since the pipeline's real outcome is already in
|
|
||||||
//! state and stdout.
|
|
||||||
|
|
||||||
use std::path::Path;
|
|
||||||
use std::process::Command;
|
|
||||||
|
|
||||||
/// What happened to a package that the operator should hear about.
|
|
||||||
#[derive(Debug, Clone, Copy, PartialEq, Eq)]
|
|
||||||
pub enum Event<'a> {
|
|
||||||
/// Tier 4-6: verified, waiting on `pkgwatch review <name> --approve`.
|
|
||||||
NeedsReview { name: &'a str, version: &'a str },
|
|
||||||
/// Tier 1-3: built and added to the local repo, waiting on `pacman -Syu`.
|
|
||||||
Published { name: &'a str, version: &'a str },
|
|
||||||
}
|
|
||||||
|
|
||||||
/// (summary, body) for `event` — pure, so the wording is unit-testable
|
|
||||||
/// without a notification daemon.
|
|
||||||
fn message(event: Event<'_>) -> (String, String) {
|
|
||||||
match event {
|
|
||||||
Event::NeedsReview { name, version } => (
|
|
||||||
format!("{name} {version} needs review"),
|
|
||||||
format!("Run `pkgwatch review {name} --approve`, then `sudo pacman -Syu`."),
|
|
||||||
),
|
|
||||||
Event::Published { name, version } => (
|
|
||||||
format!("{name} {version} published"),
|
|
||||||
"Run `sudo pacman -Syu` to install it.".to_string(),
|
|
||||||
),
|
|
||||||
}
|
|
||||||
}
|
|
||||||
|
|
||||||
/// Best-effort desktop notification via the real `notify-send`; never
|
|
||||||
/// fails the caller.
|
|
||||||
pub fn notify(event: Event<'_>) {
|
|
||||||
notify_with(Path::new("notify-send"), event);
|
|
||||||
}
|
|
||||||
|
|
||||||
/// `notify_send_bin` is injectable for the same reason as
|
|
||||||
/// `publisher::publish_with`'s `repo_add_bin`.
|
|
||||||
fn notify_with(notify_send_bin: &Path, event: Event<'_>) {
|
|
||||||
let (summary, body) = message(event);
|
|
||||||
let result = Command::new(notify_send_bin)
|
|
||||||
.args(["--app-name=pkgwatch", "--", &summary, &body])
|
|
||||||
.status();
|
|
||||||
match result {
|
|
||||||
Ok(status) if status.success() => {}
|
|
||||||
Ok(status) => eprintln!(" warning: notify-send exited with {status}"),
|
|
||||||
Err(err) => eprintln!(" warning: could not run notify-send: {err}"),
|
|
||||||
}
|
|
||||||
}
|
|
||||||
|
|
||||||
#[cfg(test)]
|
|
||||||
mod tests {
|
|
||||||
use super::*;
|
|
||||||
use crate::test_support::write_executable_script;
|
|
||||||
|
|
||||||
#[test]
|
|
||||||
fn message_for_review_names_the_approve_command() {
|
|
||||||
let (summary, body) = message(Event::NeedsReview {
|
|
||||||
name: "claude-code",
|
|
||||||
version: "v2.1.278",
|
|
||||||
});
|
|
||||||
assert_eq!(summary, "claude-code v2.1.278 needs review");
|
|
||||||
assert!(body.contains("pkgwatch review claude-code --approve"));
|
|
||||||
}
|
|
||||||
|
|
||||||
#[test]
|
|
||||||
fn message_for_publish_points_at_pacman() {
|
|
||||||
let (summary, body) = message(Event::Published {
|
|
||||||
name: "uv",
|
|
||||||
version: "0.12.17",
|
|
||||||
});
|
|
||||||
assert_eq!(summary, "uv 0.12.17 published");
|
|
||||||
assert!(body.contains("pacman -Syu"));
|
|
||||||
}
|
|
||||||
|
|
||||||
#[test]
|
|
||||||
fn notify_invokes_binary_with_summary_and_body() {
|
|
||||||
let stub_dir = tempfile::tempdir().unwrap();
|
|
||||||
let log_path = stub_dir.path().join("invoked_with.txt");
|
|
||||||
let stub = write_executable_script(
|
|
||||||
stub_dir.path(),
|
|
||||||
"fake-notify-send",
|
|
||||||
&format!("printf '%s\\n' \"$@\" > {}", log_path.display()),
|
|
||||||
);
|
|
||||||
|
|
||||||
notify_with(
|
|
||||||
&stub,
|
|
||||||
Event::Published {
|
|
||||||
name: "uv",
|
|
||||||
version: "0.12.17",
|
|
||||||
},
|
|
||||||
);
|
|
||||||
|
|
||||||
let invoked_with = std::fs::read_to_string(&log_path).unwrap();
|
|
||||||
assert!(invoked_with.contains("uv 0.12.17 published"));
|
|
||||||
}
|
|
||||||
|
|
||||||
#[test]
|
|
||||||
fn notify_survives_missing_binary() {
|
|
||||||
notify_with(
|
|
||||||
Path::new("/nonexistent/notify-send"),
|
|
||||||
Event::NeedsReview {
|
|
||||||
name: "x",
|
|
||||||
version: "1",
|
|
||||||
},
|
|
||||||
);
|
|
||||||
}
|
|
||||||
}
|
|
||||||
184
src/paths.rs
184
src/paths.rs
|
|
@ -1,184 +0,0 @@
|
||||||
//! Decides where pkgwatch's config, state, and work directories live on
|
|
||||||
//! disk — the only module that reads the environment to answer that; every
|
|
||||||
//! other module takes the directories it needs as parameters. (The pacman
|
|
||||||
//! repo dir, `pipeline::custom_repo_dir`, is resolved separately.)
|
|
||||||
//!
|
|
||||||
//! These follow the XDG base-directory spec instead of the current working
|
|
||||||
//! directory, so an installed `/usr/bin/pkgwatch` behaves the same
|
|
||||||
//! wherever it's launched from (a systemd unit, a shell, another checkout)
|
|
||||||
//! instead of only working from inside the repo.
|
|
||||||
|
|
||||||
use anyhow::{Context, Result};
|
|
||||||
use std::path::{Path, PathBuf};
|
|
||||||
|
|
||||||
const APP_DIR: &str = "pkgwatch";
|
|
||||||
|
|
||||||
#[derive(Debug, PartialEq, Eq)]
|
|
||||||
pub struct Paths {
|
|
||||||
/// `*.toml` package declarations. Config: hand-edited, worth backing up.
|
|
||||||
pub packages_dir: PathBuf,
|
|
||||||
/// Last-published/pending versions. State: small, but losing it makes
|
|
||||||
/// every package look new, so it isn't cache.
|
|
||||||
pub state_dir: PathBuf,
|
|
||||||
/// Downloaded artifacts and build trees. Cache: safe to delete.
|
|
||||||
pub work_dir: PathBuf,
|
|
||||||
}
|
|
||||||
|
|
||||||
impl Paths {
|
|
||||||
pub fn from_env() -> Result<Self> {
|
|
||||||
Self::resolve(|key| std::env::var(key).ok())
|
|
||||||
}
|
|
||||||
|
|
||||||
/// `getenv` is injectable so tests don't mutate the process-global
|
|
||||||
/// environment, which would race with `cargo test`'s parallel threads.
|
|
||||||
///
|
|
||||||
/// Each directory has a pkgwatch-specific override, then the matching
|
|
||||||
/// XDG variable, then the XDG default under `$HOME`; empty counts as
|
|
||||||
/// unset. The overrides exist for dry runs against scratch
|
|
||||||
/// directories, like `PKGWATCH_REPO_DIR` does for the pacman repo, so
|
|
||||||
/// they're used verbatim. The XDG variables and `$HOME` must be
|
|
||||||
/// absolute: the spec says to ignore a relative XDG value, and honoring
|
|
||||||
/// one would bring back the cwd dependence this module exists to remove.
|
|
||||||
fn resolve(getenv: impl Fn(&str) -> Option<String>) -> Result<Self> {
|
|
||||||
let base = |override_var: &str, xdg_var: &str, home_subpath: &str| -> Result<PathBuf> {
|
|
||||||
if let Some(dir) = non_empty(&getenv, override_var) {
|
|
||||||
return Ok(PathBuf::from(dir));
|
|
||||||
}
|
|
||||||
if let Some(dir) = absolute(&getenv, xdg_var) {
|
|
||||||
return Ok(Path::new(&dir).join(APP_DIR));
|
|
||||||
}
|
|
||||||
let home = absolute(&getenv, "HOME").context("HOME is not set to an absolute path")?;
|
|
||||||
Ok(Path::new(&home).join(home_subpath).join(APP_DIR))
|
|
||||||
};
|
|
||||||
|
|
||||||
Ok(Self {
|
|
||||||
packages_dir: base("PKGWATCH_CONFIG_DIR", "XDG_CONFIG_HOME", ".config")?
|
|
||||||
.join("packages.d"),
|
|
||||||
state_dir: base("PKGWATCH_STATE_DIR", "XDG_STATE_HOME", ".local/state")?,
|
|
||||||
work_dir: base("PKGWATCH_WORK_DIR", "XDG_CACHE_HOME", ".cache")?,
|
|
||||||
})
|
|
||||||
}
|
|
||||||
}
|
|
||||||
|
|
||||||
/// The XDG spec says an empty variable must be treated as unset.
|
|
||||||
fn non_empty(getenv: &impl Fn(&str) -> Option<String>, key: &str) -> Option<String> {
|
|
||||||
getenv(key).filter(|v| !v.is_empty())
|
|
||||||
}
|
|
||||||
|
|
||||||
/// Like `non_empty`, but also drops relative values (an empty string isn't
|
|
||||||
/// absolute either, so this subsumes the empty check).
|
|
||||||
fn absolute(getenv: &impl Fn(&str) -> Option<String>, key: &str) -> Option<String> {
|
|
||||||
getenv(key).filter(|v| Path::new(v).is_absolute())
|
|
||||||
}
|
|
||||||
|
|
||||||
#[cfg(test)]
|
|
||||||
mod tests {
|
|
||||||
use super::*;
|
|
||||||
use std::collections::HashMap;
|
|
||||||
|
|
||||||
fn env(pairs: &[(&str, &str)]) -> impl Fn(&str) -> Option<String> {
|
|
||||||
let map: HashMap<String, String> = pairs
|
|
||||||
.iter()
|
|
||||||
.map(|(k, v)| (k.to_string(), v.to_string()))
|
|
||||||
.collect();
|
|
||||||
move |key| map.get(key).cloned()
|
|
||||||
}
|
|
||||||
|
|
||||||
#[test]
|
|
||||||
fn defaults_live_under_home() {
|
|
||||||
let paths = Paths::resolve(env(&[("HOME", "/home/u")])).unwrap();
|
|
||||||
assert_eq!(
|
|
||||||
paths,
|
|
||||||
Paths {
|
|
||||||
packages_dir: "/home/u/.config/pkgwatch/packages.d".into(),
|
|
||||||
state_dir: "/home/u/.local/state/pkgwatch".into(),
|
|
||||||
work_dir: "/home/u/.cache/pkgwatch".into(),
|
|
||||||
}
|
|
||||||
);
|
|
||||||
}
|
|
||||||
|
|
||||||
#[test]
|
|
||||||
fn xdg_variables_override_home_defaults() {
|
|
||||||
let paths = Paths::resolve(env(&[
|
|
||||||
("HOME", "/home/u"),
|
|
||||||
("XDG_CONFIG_HOME", "/xdg/config"),
|
|
||||||
("XDG_STATE_HOME", "/xdg/state"),
|
|
||||||
("XDG_CACHE_HOME", "/xdg/cache"),
|
|
||||||
]))
|
|
||||||
.unwrap();
|
|
||||||
assert_eq!(
|
|
||||||
paths.packages_dir,
|
|
||||||
Path::new("/xdg/config/pkgwatch/packages.d")
|
|
||||||
);
|
|
||||||
assert_eq!(paths.state_dir, Path::new("/xdg/state/pkgwatch"));
|
|
||||||
assert_eq!(paths.work_dir, Path::new("/xdg/cache/pkgwatch"));
|
|
||||||
}
|
|
||||||
|
|
||||||
#[test]
|
|
||||||
fn pkgwatch_overrides_win_and_are_used_verbatim() {
|
|
||||||
let paths = Paths::resolve(env(&[
|
|
||||||
("HOME", "/home/u"),
|
|
||||||
("XDG_STATE_HOME", "/xdg/state"),
|
|
||||||
("PKGWATCH_CONFIG_DIR", "/scratch/cfg"),
|
|
||||||
("PKGWATCH_STATE_DIR", "/scratch/state"),
|
|
||||||
("PKGWATCH_WORK_DIR", "/scratch/work"),
|
|
||||||
]))
|
|
||||||
.unwrap();
|
|
||||||
// No `pkgwatch/` suffix appended to an explicit override.
|
|
||||||
assert_eq!(paths.packages_dir, Path::new("/scratch/cfg/packages.d"));
|
|
||||||
assert_eq!(paths.state_dir, Path::new("/scratch/state"));
|
|
||||||
assert_eq!(paths.work_dir, Path::new("/scratch/work"));
|
|
||||||
}
|
|
||||||
|
|
||||||
#[test]
|
|
||||||
fn empty_variables_are_treated_as_unset() {
|
|
||||||
let paths = Paths::resolve(env(&[
|
|
||||||
("HOME", "/home/u"),
|
|
||||||
("XDG_STATE_HOME", ""),
|
|
||||||
("PKGWATCH_WORK_DIR", ""),
|
|
||||||
]))
|
|
||||||
.unwrap();
|
|
||||||
assert_eq!(paths.state_dir, Path::new("/home/u/.local/state/pkgwatch"));
|
|
||||||
assert_eq!(paths.work_dir, Path::new("/home/u/.cache/pkgwatch"));
|
|
||||||
}
|
|
||||||
|
|
||||||
#[test]
|
|
||||||
fn relative_xdg_variables_are_ignored() {
|
|
||||||
let paths = Paths::resolve(env(&[
|
|
||||||
("HOME", "/home/u"),
|
|
||||||
("XDG_CONFIG_HOME", "rel/config"),
|
|
||||||
("XDG_STATE_HOME", "./state"),
|
|
||||||
("XDG_CACHE_HOME", "cache"),
|
|
||||||
]))
|
|
||||||
.unwrap();
|
|
||||||
assert_eq!(
|
|
||||||
paths.packages_dir,
|
|
||||||
Path::new("/home/u/.config/pkgwatch/packages.d")
|
|
||||||
);
|
|
||||||
assert_eq!(paths.state_dir, Path::new("/home/u/.local/state/pkgwatch"));
|
|
||||||
assert_eq!(paths.work_dir, Path::new("/home/u/.cache/pkgwatch"));
|
|
||||||
}
|
|
||||||
|
|
||||||
#[test]
|
|
||||||
fn relative_home_is_an_error() {
|
|
||||||
let err = Paths::resolve(env(&[("HOME", "relative/home")])).unwrap_err();
|
|
||||||
assert!(err.to_string().contains("HOME"));
|
|
||||||
}
|
|
||||||
|
|
||||||
#[test]
|
|
||||||
fn errors_when_nothing_locates_home() {
|
|
||||||
let err = Paths::resolve(env(&[])).unwrap_err();
|
|
||||||
assert!(err.to_string().contains("HOME"));
|
|
||||||
}
|
|
||||||
|
|
||||||
#[test]
|
|
||||||
fn no_home_needed_when_every_dir_is_overridden() {
|
|
||||||
let paths = Paths::resolve(env(&[
|
|
||||||
("PKGWATCH_CONFIG_DIR", "/c"),
|
|
||||||
("PKGWATCH_STATE_DIR", "/s"),
|
|
||||||
("PKGWATCH_WORK_DIR", "/w"),
|
|
||||||
]))
|
|
||||||
.unwrap();
|
|
||||||
assert_eq!(paths.state_dir, Path::new("/s"));
|
|
||||||
}
|
|
||||||
}
|
|
||||||
215
src/pipeline.rs
215
src/pipeline.rs
|
|
@ -9,16 +9,17 @@ use crate::builder;
|
||||||
use crate::checker;
|
use crate::checker;
|
||||||
use crate::config::{self, Package};
|
use crate::config::{self, Package};
|
||||||
use crate::fetcher::{self, DownloadedAsset};
|
use crate::fetcher::{self, DownloadedAsset};
|
||||||
use crate::notifier::{self, Event};
|
use crate::github::GithubEndpoints;
|
||||||
use crate::paths::Paths;
|
|
||||||
use crate::publisher;
|
use crate::publisher;
|
||||||
use crate::release_source::{self, ReleaseSource};
|
|
||||||
use crate::sanity;
|
use crate::sanity;
|
||||||
use crate::state;
|
use crate::state;
|
||||||
use crate::verifier::{self, VerificationResult};
|
use crate::verifier::{self, VerificationResult};
|
||||||
use anyhow::{Context, Result, bail};
|
use anyhow::{Context, Result, bail};
|
||||||
use std::path::{Path, PathBuf};
|
use std::path::{Path, PathBuf};
|
||||||
|
|
||||||
|
const PACKAGES_DIR: &str = "packages.d";
|
||||||
|
const STATE_DIR: &str = "state";
|
||||||
|
const WORK_DIR: &str = "work";
|
||||||
/// Not a repo pkgwatch invents: this is the existing, already-registered
|
/// Not a repo pkgwatch invents: this is the existing, already-registered
|
||||||
/// local pacman repo on this box (see `[custom]` in /etc/pacman.conf and
|
/// local pacman repo on this box (see `[custom]` in /etc/pacman.conf and
|
||||||
/// its `Server = file://...` line). pkgwatch adds packages to it; it does
|
/// its `Server = file://...` line). pkgwatch adds packages to it; it does
|
||||||
|
|
@ -42,28 +43,14 @@ fn custom_repo_dir() -> Result<PathBuf> {
|
||||||
Ok(Path::new(&home).join(CUSTOM_REPO_SUBPATH))
|
Ok(Path::new(&home).join(CUSTOM_REPO_SUBPATH))
|
||||||
}
|
}
|
||||||
|
|
||||||
/// Adds a hint to the bare "No such file" a missing config dir would give:
|
|
||||||
/// the dir is no longer relative to the cwd, so a checkout's `packages.d/`
|
|
||||||
/// isn't picked up on its own (see docs/SPEC.md > Paths).
|
|
||||||
fn load_packages(packages_dir: &Path) -> Result<Vec<(String, Package)>> {
|
|
||||||
config::load_packages_dir(packages_dir).with_context(|| {
|
|
||||||
format!(
|
|
||||||
"no package config at {} (set PKGWATCH_CONFIG_DIR to the directory containing \
|
|
||||||
packages.d, or see docs/SPEC.md > Paths for moving a checkout's packages.d/ there)",
|
|
||||||
packages_dir.display()
|
|
||||||
)
|
|
||||||
})
|
|
||||||
}
|
|
||||||
|
|
||||||
pub fn run_check() -> Result<()> {
|
pub fn run_check() -> Result<()> {
|
||||||
let client = build_client()?;
|
let client = build_client()?;
|
||||||
let Paths {
|
let endpoints = GithubEndpoints::default();
|
||||||
packages_dir,
|
let packages_dir = Path::new(PACKAGES_DIR);
|
||||||
state_dir,
|
let state_dir = Path::new(STATE_DIR);
|
||||||
work_dir,
|
let work_dir = Path::new(WORK_DIR);
|
||||||
} = Paths::from_env()?;
|
|
||||||
|
|
||||||
let packages = load_packages(&packages_dir)?;
|
let packages = config::load_packages_dir(packages_dir)?;
|
||||||
if packages.is_empty() {
|
if packages.is_empty() {
|
||||||
println!("no packages configured under {}/", packages_dir.display());
|
println!("no packages configured under {}/", packages_dir.display());
|
||||||
return Ok(());
|
return Ok(());
|
||||||
|
|
@ -72,10 +59,7 @@ pub fn run_check() -> Result<()> {
|
||||||
let mut any_failed = false;
|
let mut any_failed = false;
|
||||||
for (name, pkg) in &packages {
|
for (name, pkg) in &packages {
|
||||||
println!("== {name} ({}) ==", pkg.repo);
|
println!("== {name} ({}) ==", pkg.repo);
|
||||||
let result = release_source::for_package(pkg).and_then(|host| {
|
if let Err(err) = process_package(&client, &endpoints, state_dir, work_dir, name, pkg) {
|
||||||
process_package(&client, host.as_ref(), &state_dir, &work_dir, name, pkg)
|
|
||||||
});
|
|
||||||
if let Err(err) = result {
|
|
||||||
eprintln!(" error: {err:#}");
|
eprintln!(" error: {err:#}");
|
||||||
any_failed = true;
|
any_failed = true;
|
||||||
}
|
}
|
||||||
|
|
@ -120,13 +104,13 @@ fn decide_tier_action(tier: u8, passed: bool, already_pending_this_version: bool
|
||||||
|
|
||||||
fn process_package(
|
fn process_package(
|
||||||
client: &reqwest::blocking::Client,
|
client: &reqwest::blocking::Client,
|
||||||
host: &dyn ReleaseSource,
|
endpoints: &GithubEndpoints,
|
||||||
state_dir: &Path,
|
state_dir: &Path,
|
||||||
work_dir: &Path,
|
work_dir: &Path,
|
||||||
name: &str,
|
name: &str,
|
||||||
pkg: &Package,
|
pkg: &Package,
|
||||||
) -> Result<()> {
|
) -> Result<()> {
|
||||||
let latest = host.latest_release(client, &pkg.repo)?;
|
let latest = checker::latest_github_release(client, endpoints, &pkg.repo)?;
|
||||||
let last_seen = state::load_last_version(state_dir, name);
|
let last_seen = state::load_last_version(state_dir, name);
|
||||||
if last_seen.as_deref() == Some(latest.as_str()) {
|
if last_seen.as_deref() == Some(latest.as_str()) {
|
||||||
println!(" up to date at {latest}");
|
println!(" up to date at {latest}");
|
||||||
|
|
@ -134,7 +118,7 @@ fn process_package(
|
||||||
}
|
}
|
||||||
println!(" new version detected: {latest} (previously: {last_seen:?})");
|
println!(" new version detected: {latest} (previously: {last_seen:?})");
|
||||||
|
|
||||||
let fetched = fetch_and_verify(client, host, work_dir, name, pkg, &latest)?;
|
let fetched = fetch_and_verify(client, endpoints, work_dir, name, pkg, &latest)?;
|
||||||
println!(" fetched {}", fetched.asset.path.display());
|
println!(" fetched {}", fetched.asset.path.display());
|
||||||
println!(
|
println!(
|
||||||
" verification (tier {}): {} — {}",
|
" verification (tier {}): {} — {}",
|
||||||
|
|
@ -167,10 +151,6 @@ fn process_package(
|
||||||
state::save_last_version(state_dir, name, &latest)?;
|
state::save_last_version(state_dir, name, &latest)?;
|
||||||
state::clear_pending_version(state_dir, name)?;
|
state::clear_pending_version(state_dir, name)?;
|
||||||
println!(" published {name} {latest}");
|
println!(" published {name} {latest}");
|
||||||
notifier::notify(Event::Published {
|
|
||||||
name,
|
|
||||||
version: &latest,
|
|
||||||
});
|
|
||||||
}
|
}
|
||||||
TierAction::StillPending => {
|
TierAction::StillPending => {
|
||||||
println!(" tier 4-6 pass: still pending review (`pkgwatch review` to see it)");
|
println!(" tier 4-6 pass: still pending review (`pkgwatch review` to see it)");
|
||||||
|
|
@ -185,10 +165,6 @@ fn process_package(
|
||||||
" tier 4-6 pass: flagged for human review (`pkgwatch review` to approve)"
|
" tier 4-6 pass: flagged for human review (`pkgwatch review` to approve)"
|
||||||
),
|
),
|
||||||
}
|
}
|
||||||
notifier::notify(Event::NeedsReview {
|
|
||||||
name,
|
|
||||||
version: &latest,
|
|
||||||
});
|
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
Ok(())
|
Ok(())
|
||||||
|
|
@ -207,7 +183,7 @@ struct FetchVerifyResult {
|
||||||
/// trusting a possibly-stale flag from an earlier run).
|
/// trusting a possibly-stale flag from an earlier run).
|
||||||
fn fetch_and_verify(
|
fn fetch_and_verify(
|
||||||
client: &reqwest::blocking::Client,
|
client: &reqwest::blocking::Client,
|
||||||
host: &dyn ReleaseSource,
|
endpoints: &GithubEndpoints,
|
||||||
work_dir: &Path,
|
work_dir: &Path,
|
||||||
name: &str,
|
name: &str,
|
||||||
pkg: &Package,
|
pkg: &Package,
|
||||||
|
|
@ -217,11 +193,10 @@ fn fetch_and_verify(
|
||||||
let asset_name = pkg.asset_pattern.replace("{version}", &version);
|
let asset_name = pkg.asset_pattern.replace("{version}", &version);
|
||||||
let dest_dir = work_dir.join(name).join(tag);
|
let dest_dir = work_dir.join(name).join(tag);
|
||||||
|
|
||||||
let api = host.api();
|
let asset = fetcher::download_asset(client, endpoints, &pkg.repo, tag, &asset_name, &dest_dir)?;
|
||||||
let asset = fetcher::download_asset(client, api, &pkg.repo, tag, &asset_name, &dest_dir)?;
|
|
||||||
let verification = verifier::verify(
|
let verification = verifier::verify(
|
||||||
client,
|
client,
|
||||||
api,
|
endpoints,
|
||||||
&pkg.verification,
|
&pkg.verification,
|
||||||
&pkg.repo,
|
&pkg.repo,
|
||||||
tag,
|
tag,
|
||||||
|
|
@ -283,18 +258,16 @@ fn build_and_publish(
|
||||||
}
|
}
|
||||||
|
|
||||||
pub fn run_review(args: &[String]) -> Result<()> {
|
pub fn run_review(args: &[String]) -> Result<()> {
|
||||||
let Paths {
|
let packages_dir = Path::new(PACKAGES_DIR);
|
||||||
packages_dir,
|
let state_dir = Path::new(STATE_DIR);
|
||||||
state_dir,
|
let work_dir = Path::new(WORK_DIR);
|
||||||
work_dir,
|
let packages = config::load_packages_dir(packages_dir)?;
|
||||||
} = Paths::from_env()?;
|
|
||||||
let packages = load_packages(&packages_dir)?;
|
|
||||||
|
|
||||||
match args {
|
match args {
|
||||||
[] => {
|
[] => {
|
||||||
let mut any = false;
|
let mut any = false;
|
||||||
for (name, _) in &packages {
|
for (name, _) in &packages {
|
||||||
if let Some(pending) = state::load_pending_version(&state_dir, name) {
|
if let Some(pending) = state::load_pending_version(state_dir, name) {
|
||||||
println!(
|
println!(
|
||||||
"{name}: {pending} pending review (run `pkgwatch review {name} --approve`)"
|
"{name}: {pending} pending review (run `pkgwatch review {name} --approve`)"
|
||||||
);
|
);
|
||||||
|
|
@ -310,9 +283,9 @@ pub fn run_review(args: &[String]) -> Result<()> {
|
||||||
let (_, pkg) = packages.iter().find(|(n, _)| n == name).with_context(|| {
|
let (_, pkg) = packages.iter().find(|(n, _)| n == name).with_context(|| {
|
||||||
format!("no package named '{name}' in {}/", packages_dir.display())
|
format!("no package named '{name}' in {}/", packages_dir.display())
|
||||||
})?;
|
})?;
|
||||||
let tag = state::load_pending_version(&state_dir, name)
|
let tag = state::load_pending_version(state_dir, name)
|
||||||
.with_context(|| format!("'{name}' has no pending review"))?;
|
.with_context(|| format!("'{name}' has no pending review"))?;
|
||||||
approve(&state_dir, &work_dir, name, pkg, &tag)
|
approve(state_dir, work_dir, name, pkg, &tag)
|
||||||
}
|
}
|
||||||
_ => bail!("usage: pkgwatch review [<name> --approve]"),
|
_ => bail!("usage: pkgwatch review [<name> --approve]"),
|
||||||
}
|
}
|
||||||
|
|
@ -320,11 +293,11 @@ pub fn run_review(args: &[String]) -> Result<()> {
|
||||||
|
|
||||||
fn approve(state_dir: &Path, work_dir: &Path, name: &str, pkg: &Package, tag: &str) -> Result<()> {
|
fn approve(state_dir: &Path, work_dir: &Path, name: &str, pkg: &Package, tag: &str) -> Result<()> {
|
||||||
let client = build_client()?;
|
let client = build_client()?;
|
||||||
let host = release_source::for_package(pkg)?;
|
let endpoints = GithubEndpoints::default();
|
||||||
|
|
||||||
// Re-verify rather than trusting the earlier flag: the artifact at
|
// Re-verify rather than trusting the earlier flag: the artifact at
|
||||||
// this tag could in principle have changed since it was queued.
|
// this tag could in principle have changed since it was queued.
|
||||||
let fetched = fetch_and_verify(&client, host.as_ref(), work_dir, name, pkg, tag)?;
|
let fetched = fetch_and_verify(&client, &endpoints, work_dir, name, pkg, tag)?;
|
||||||
if !fetched.verification.passed {
|
if !fetched.verification.passed {
|
||||||
bail!(
|
bail!(
|
||||||
"re-verification failed on approve: {}",
|
"re-verification failed on approve: {}",
|
||||||
|
|
@ -346,8 +319,6 @@ fn approve(state_dir: &Path, work_dir: &Path, name: &str, pkg: &Package, tag: &s
|
||||||
#[cfg(test)]
|
#[cfg(test)]
|
||||||
mod tests {
|
mod tests {
|
||||||
use super::*;
|
use super::*;
|
||||||
use crate::release_source::{ForgejoEndpoints, GithubEndpoints};
|
|
||||||
use crate::test_support::same_origin_package;
|
|
||||||
|
|
||||||
#[test]
|
#[test]
|
||||||
fn decide_tier_action_failed_verification_overrides_everything() {
|
fn decide_tier_action_failed_verification_overrides_everything() {
|
||||||
|
|
@ -379,67 +350,6 @@ mod tests {
|
||||||
assert_eq!(decide_tier_action(4, true, true), TierAction::StillPending);
|
assert_eq!(decide_tier_action(4, true, true), TierAction::StillPending);
|
||||||
}
|
}
|
||||||
|
|
||||||
/// Mocks the release-tag, asset, and checksum endpoints for `o/r@v1.0.0`
|
|
||||||
/// with a checksum that doesn't match the asset, so verification fails.
|
|
||||||
/// These are identical on GitHub and Forgejo (see `ReleaseSource::api`); only
|
|
||||||
/// how the latest tag is found differs per test. Returned mocks must
|
|
||||||
/// stay alive for the test's duration.
|
|
||||||
fn mock_release_with_bad_checksum(server: &mut mockito::ServerGuard) -> Vec<mockito::Mock> {
|
|
||||||
let asset_url = format!("{}/download/thing.tar.gz", server.url());
|
|
||||||
let sums_url = format!("{}/download/SHA256SUMS", server.url());
|
|
||||||
let release_body = format!(
|
|
||||||
r#"{{"assets": [
|
|
||||||
{{"name": "thing.tar.gz", "browser_download_url": "{asset_url}"}},
|
|
||||||
{{"name": "SHA256SUMS", "browser_download_url": "{sums_url}"}}
|
|
||||||
]}}"#
|
|
||||||
);
|
|
||||||
vec![
|
|
||||||
server
|
|
||||||
.mock("GET", "/repos/o/r/releases/tags/v1.0.0")
|
|
||||||
.with_status(200)
|
|
||||||
.with_body(release_body)
|
|
||||||
.create(),
|
|
||||||
server
|
|
||||||
.mock("GET", "/download/thing.tar.gz")
|
|
||||||
.with_status(200)
|
|
||||||
.with_body(b"artifact-bytes".as_slice())
|
|
||||||
.create(),
|
|
||||||
// Wrong hash for "artifact-bytes" — forces a verification failure.
|
|
||||||
server
|
|
||||||
.mock("GET", "/download/SHA256SUMS")
|
|
||||||
.with_status(200)
|
|
||||||
.with_body(
|
|
||||||
"0000000000000000000000000000000000000000000000000000000000000000 thing.tar.gz\n",
|
|
||||||
)
|
|
||||||
.create(),
|
|
||||||
]
|
|
||||||
}
|
|
||||||
|
|
||||||
/// Runs `process_package` for a package whose checksum is wrong and
|
|
||||||
/// asserts it errors with "verification failed" while leaving no trace
|
|
||||||
/// in state.
|
|
||||||
fn assert_verification_failure_is_an_error(host: &dyn ReleaseSource, pkg: &Package) {
|
|
||||||
let client = reqwest::blocking::Client::new();
|
|
||||||
let state_dir = tempfile::tempdir().unwrap();
|
|
||||||
let work_dir = tempfile::tempdir().unwrap();
|
|
||||||
|
|
||||||
let err = process_package(
|
|
||||||
&client,
|
|
||||||
host,
|
|
||||||
state_dir.path(),
|
|
||||||
work_dir.path(),
|
|
||||||
"thing",
|
|
||||||
pkg,
|
|
||||||
)
|
|
||||||
.unwrap_err();
|
|
||||||
|
|
||||||
assert!(err.to_string().contains("verification failed"));
|
|
||||||
// Neither published nor queued for review — a failed verification
|
|
||||||
// shouldn't leave any trace in state.
|
|
||||||
assert_eq!(state::load_last_version(state_dir.path(), "thing"), None);
|
|
||||||
assert_eq!(state::load_pending_version(state_dir.path(), "thing"), None);
|
|
||||||
}
|
|
||||||
|
|
||||||
/// Regression test for the exit-code gap this PR fixes: a verification
|
/// Regression test for the exit-code gap this PR fixes: a verification
|
||||||
/// failure previously returned `Ok(())` from `process_package`, so
|
/// failure previously returned `Ok(())` from `process_package`, so
|
||||||
/// `run_check` never counted it as a failure and the process exited 0
|
/// `run_check` never counted it as a failure and the process exited 0
|
||||||
|
|
@ -450,10 +360,11 @@ mod tests {
|
||||||
#[test]
|
#[test]
|
||||||
fn process_package_returns_err_on_verification_failure() {
|
fn process_package_returns_err_on_verification_failure() {
|
||||||
let mut server = mockito::Server::new();
|
let mut server = mockito::Server::new();
|
||||||
let github = GithubEndpoints {
|
let endpoints = GithubEndpoints {
|
||||||
web: server.url(),
|
web: server.url(),
|
||||||
api: server.url(),
|
api: server.url(),
|
||||||
};
|
};
|
||||||
|
|
||||||
let feed = format!(
|
let feed = format!(
|
||||||
r#"<feed><link rel="alternate" href="{}/o/r/releases/tag/v1.0.0"/></feed>"#,
|
r#"<feed><link rel="alternate" href="{}/o/r/releases/tag/v1.0.0"/></feed>"#,
|
||||||
server.url()
|
server.url()
|
||||||
|
|
@ -463,27 +374,63 @@ mod tests {
|
||||||
.with_status(200)
|
.with_status(200)
|
||||||
.with_body(feed)
|
.with_body(feed)
|
||||||
.create();
|
.create();
|
||||||
let _release_mocks = mock_release_with_bad_checksum(&mut server);
|
|
||||||
|
|
||||||
assert_verification_failure_is_an_error(&github, &same_origin_package(""));
|
let asset_url = format!("{}/download/thing.tar.gz", server.url());
|
||||||
}
|
let sums_url = format!("{}/download/SHA256SUMS", server.url());
|
||||||
|
let release_body = format!(
|
||||||
/// Same as above but through a Forgejo source, proving the whole
|
r#"{{"assets": [
|
||||||
/// check -> fetch -> verify path works there too: the error is the
|
{{"name": "thing.tar.gz", "browser_download_url": "{asset_url}"}},
|
||||||
/// verification failure, not a fetch or check failure on the way to it.
|
{{"name": "SHA256SUMS", "browser_download_url": "{sums_url}"}}
|
||||||
#[test]
|
]}}"#
|
||||||
fn process_package_returns_err_on_verification_failure_via_forgejo() {
|
);
|
||||||
let mut server = mockito::Server::new();
|
let _release = server
|
||||||
let forgejo = ForgejoEndpoints { api: server.url() };
|
.mock("GET", "/repos/o/r/releases/tags/v1.0.0")
|
||||||
let _latest = server
|
|
||||||
.mock("GET", "/repos/o/r/releases/latest")
|
|
||||||
.with_status(200)
|
.with_status(200)
|
||||||
.with_body(r#"{"tag_name": "v1.0.0"}"#)
|
.with_body(release_body)
|
||||||
|
.create();
|
||||||
|
let _asset = server
|
||||||
|
.mock("GET", "/download/thing.tar.gz")
|
||||||
|
.with_status(200)
|
||||||
|
.with_body(b"artifact-bytes".as_slice())
|
||||||
|
.create();
|
||||||
|
// Wrong hash for "artifact-bytes" — forces a verification failure.
|
||||||
|
let _sums = server
|
||||||
|
.mock("GET", "/download/SHA256SUMS")
|
||||||
|
.with_status(200)
|
||||||
|
.with_body(
|
||||||
|
"0000000000000000000000000000000000000000000000000000000000000000 thing.tar.gz\n",
|
||||||
|
)
|
||||||
.create();
|
.create();
|
||||||
let _release_mocks = mock_release_with_bad_checksum(&mut server);
|
|
||||||
|
|
||||||
// The package's own `source` is irrelevant here: `forgejo` is
|
let pkg: Package = toml::from_str(
|
||||||
// hand-built to point at the mock server, bypassing `for_package`.
|
r#"
|
||||||
assert_verification_failure_is_an_error(&forgejo, &same_origin_package(""));
|
repo = "o/r"
|
||||||
|
asset_pattern = "thing.tar.gz"
|
||||||
|
[verification]
|
||||||
|
method = "same-origin-sha256"
|
||||||
|
checksum_asset_pattern = "SHA256SUMS"
|
||||||
|
"#,
|
||||||
|
)
|
||||||
|
.unwrap();
|
||||||
|
|
||||||
|
let client = reqwest::blocking::Client::new();
|
||||||
|
let state_dir = tempfile::tempdir().unwrap();
|
||||||
|
let work_dir = tempfile::tempdir().unwrap();
|
||||||
|
|
||||||
|
let err = process_package(
|
||||||
|
&client,
|
||||||
|
&endpoints,
|
||||||
|
state_dir.path(),
|
||||||
|
work_dir.path(),
|
||||||
|
"thing",
|
||||||
|
&pkg,
|
||||||
|
)
|
||||||
|
.unwrap_err();
|
||||||
|
|
||||||
|
assert!(err.to_string().contains("verification failed"));
|
||||||
|
// Neither published nor queued for review — a failed verification
|
||||||
|
// shouldn't leave any trace in state.
|
||||||
|
assert_eq!(state::load_last_version(state_dir.path(), "thing"), None);
|
||||||
|
assert_eq!(state::load_pending_version(state_dir.path(), "thing"), None);
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
|
|
|
||||||
|
|
@ -1,20 +0,0 @@
|
||||||
//! The `ReleaseSource` trait: what the pipeline needs from a hosting
|
|
||||||
//! service a package's releases are published on. Each host implements it
|
|
||||||
//! in its own file next to this one, so per-host logic never accumulates
|
|
||||||
//! here.
|
|
||||||
|
|
||||||
use anyhow::Result;
|
|
||||||
|
|
||||||
/// `Debug` so a `Box<dyn ReleaseSource>` can sit in a `Result` that tests
|
|
||||||
/// unwrap.
|
|
||||||
pub trait ReleaseSource: std::fmt::Debug {
|
|
||||||
/// The latest release tag for `repo`.
|
|
||||||
fn latest_release(&self, client: &reqwest::blocking::Client, repo: &str) -> Result<String>;
|
|
||||||
|
|
||||||
/// The releases API root, which `fetcher` and `verifier` build their
|
|
||||||
/// own paths under. GitHub and Forgejo both serve
|
|
||||||
/// `/repos/{owner}/{repo}/releases/tags/{tag}` there with the same
|
|
||||||
/// `assets[].{name, browser_download_url}` shape, which is why those
|
|
||||||
/// stages need only this and not the source itself.
|
|
||||||
fn api(&self) -> &str;
|
|
||||||
}
|
|
||||||
|
|
@ -1,112 +0,0 @@
|
||||||
//! A Forgejo (or Gitea) instance as a release source: its API root, and
|
|
||||||
//! how to find a repo's latest release there.
|
|
||||||
|
|
||||||
use super::ReleaseSource;
|
|
||||||
use anyhow::{Context, Result, bail};
|
|
||||||
use serde::Deserialize;
|
|
||||||
|
|
||||||
#[derive(Debug, Clone)]
|
|
||||||
pub struct ForgejoEndpoints {
|
|
||||||
/// The instance's API root, i.e. `<base_url>/api/v1`.
|
|
||||||
pub api: String,
|
|
||||||
}
|
|
||||||
|
|
||||||
impl ForgejoEndpoints {
|
|
||||||
/// `base_url` is the instance's web root, e.g.
|
|
||||||
/// `https://code.austinschaefer.com`; a trailing slash is tolerated.
|
|
||||||
pub fn from_base_url(base_url: &str) -> Self {
|
|
||||||
Self {
|
|
||||||
api: format!("{}/api/v1", base_url.trim_end_matches('/')),
|
|
||||||
}
|
|
||||||
}
|
|
||||||
}
|
|
||||||
|
|
||||||
impl ReleaseSource for ForgejoEndpoints {
|
|
||||||
/// One call, unlike GitHub's feed-then-confirm dance: Forgejo's
|
|
||||||
/// `releases/latest` already returns only the newest non-draft,
|
|
||||||
/// non-prerelease *release object*, so a stray tag with no release
|
|
||||||
/// behind it (GitHub's scaleway-cli `-dbg1` problem) can't be returned.
|
|
||||||
fn latest_release(&self, client: &reqwest::blocking::Client, repo: &str) -> Result<String> {
|
|
||||||
#[derive(Deserialize)]
|
|
||||||
struct Latest {
|
|
||||||
tag_name: String,
|
|
||||||
}
|
|
||||||
|
|
||||||
let url = format!("{}/repos/{repo}/releases/latest", self.api);
|
|
||||||
let response = client.get(&url).send()?;
|
|
||||||
// Forgejo answers 404 both for an unknown repo and for one with no
|
|
||||||
// releases yet — the common state for a project's very first
|
|
||||||
// release.
|
|
||||||
if response.status() == reqwest::StatusCode::NOT_FOUND {
|
|
||||||
bail!("no published release found at {url} (repo missing, or nothing released yet)");
|
|
||||||
}
|
|
||||||
let latest: Latest = response
|
|
||||||
.error_for_status()
|
|
||||||
.with_context(|| format!("fetching latest release from {url}"))?
|
|
||||||
.json()?;
|
|
||||||
Ok(latest.tag_name)
|
|
||||||
}
|
|
||||||
|
|
||||||
fn api(&self) -> &str {
|
|
||||||
&self.api
|
|
||||||
}
|
|
||||||
}
|
|
||||||
|
|
||||||
#[cfg(test)]
|
|
||||||
mod tests {
|
|
||||||
use super::*;
|
|
||||||
|
|
||||||
#[test]
|
|
||||||
fn from_base_url_appends_api_v1() {
|
|
||||||
let endpoints = ForgejoEndpoints::from_base_url("https://code.example.com");
|
|
||||||
assert_eq!(endpoints.api(), "https://code.example.com/api/v1");
|
|
||||||
}
|
|
||||||
|
|
||||||
#[test]
|
|
||||||
fn from_base_url_tolerates_trailing_slash() {
|
|
||||||
let endpoints = ForgejoEndpoints::from_base_url("https://code.example.com/");
|
|
||||||
assert_eq!(endpoints.api(), "https://code.example.com/api/v1");
|
|
||||||
}
|
|
||||||
|
|
||||||
#[test]
|
|
||||||
fn latest_release_returns_tag_name() {
|
|
||||||
let mut server = mockito::Server::new();
|
|
||||||
let _latest = server
|
|
||||||
.mock("GET", "/repos/o/r/releases/latest")
|
|
||||||
.with_status(200)
|
|
||||||
.with_body(r#"{"tag_name": "v0.1.0", "assets": []}"#)
|
|
||||||
.create();
|
|
||||||
|
|
||||||
let client = reqwest::blocking::Client::new();
|
|
||||||
let endpoints = ForgejoEndpoints { api: server.url() };
|
|
||||||
assert_eq!(endpoints.latest_release(&client, "o/r").unwrap(), "v0.1.0");
|
|
||||||
}
|
|
||||||
|
|
||||||
#[test]
|
|
||||||
fn latest_release_names_the_no_releases_case() {
|
|
||||||
let mut server = mockito::Server::new();
|
|
||||||
let _latest = server
|
|
||||||
.mock("GET", "/repos/o/r/releases/latest")
|
|
||||||
.with_status(404)
|
|
||||||
.create();
|
|
||||||
|
|
||||||
let client = reqwest::blocking::Client::new();
|
|
||||||
let endpoints = ForgejoEndpoints { api: server.url() };
|
|
||||||
let err = endpoints.latest_release(&client, "o/r").unwrap_err();
|
|
||||||
assert!(err.to_string().contains("no published release"));
|
|
||||||
}
|
|
||||||
|
|
||||||
#[test]
|
|
||||||
fn latest_release_surfaces_server_errors() {
|
|
||||||
let mut server = mockito::Server::new();
|
|
||||||
let _latest = server
|
|
||||||
.mock("GET", "/repos/o/r/releases/latest")
|
|
||||||
.with_status(500)
|
|
||||||
.create();
|
|
||||||
|
|
||||||
let client = reqwest::blocking::Client::new();
|
|
||||||
let endpoints = ForgejoEndpoints { api: server.url() };
|
|
||||||
let err = endpoints.latest_release(&client, "o/r").unwrap_err();
|
|
||||||
assert!(err.to_string().contains("fetching latest release"));
|
|
||||||
}
|
|
||||||
}
|
|
||||||
|
|
@ -1,166 +0,0 @@
|
||||||
//! GitHub as a release source: its endpoints, and how to find a repo's
|
|
||||||
//! latest release there.
|
|
||||||
|
|
||||||
use super::ReleaseSource;
|
|
||||||
use anyhow::{Result, bail};
|
|
||||||
use regex::Regex;
|
|
||||||
|
|
||||||
/// Base URLs for GitHub's public web host (Atom feeds, release pages) and
|
|
||||||
/// its REST API, factored out so tests can point both at a local mock
|
|
||||||
/// server instead of the real github.com/api.github.com.
|
|
||||||
#[derive(Debug, Clone)]
|
|
||||||
pub struct GithubEndpoints {
|
|
||||||
pub web: String,
|
|
||||||
pub api: String,
|
|
||||||
}
|
|
||||||
|
|
||||||
impl Default for GithubEndpoints {
|
|
||||||
fn default() -> Self {
|
|
||||||
Self {
|
|
||||||
web: "https://github.com".to_string(),
|
|
||||||
api: "https://api.github.com".to_string(),
|
|
||||||
}
|
|
||||||
}
|
|
||||||
}
|
|
||||||
|
|
||||||
impl ReleaseSource for GithubEndpoints {
|
|
||||||
/// Resolves the latest release tag for `repo` via its public Atom feed.
|
|
||||||
///
|
|
||||||
/// Deliberately not a full XML parse: the feed lists entries
|
|
||||||
/// newest-first, and each `<link rel="alternate"
|
|
||||||
/// .../releases/tag/<tag>"/>` is matched in document order. Revisit
|
|
||||||
/// with a real XML parser if GitHub's feed shape ever changes.
|
|
||||||
///
|
|
||||||
/// The feed can list a tag newer than any tag with a real Release
|
|
||||||
/// object behind it — observed on scaleway/scaleway-cli, which pushes a
|
|
||||||
/// `vX.Y.Z-dbg1` tag (no corresponding Release; `releases/tags/<tag>`
|
|
||||||
/// 404s) right after each real release, and that tag sorts newest in
|
|
||||||
/// the feed. So each candidate is confirmed against the releases API in
|
|
||||||
/// feed order, returning the first that actually resolves.
|
|
||||||
fn latest_release(&self, client: &reqwest::blocking::Client, repo: &str) -> Result<String> {
|
|
||||||
let url = format!("{}/{repo}/releases.atom", self.web);
|
|
||||||
let body = client.get(&url).send()?.error_for_status()?.text()?;
|
|
||||||
|
|
||||||
let re = Regex::new(r#"releases/tag/([^"]+)""#)?;
|
|
||||||
let mut candidates = re
|
|
||||||
.captures_iter(&body)
|
|
||||||
.map(|caps| caps[1].to_string())
|
|
||||||
.peekable();
|
|
||||||
if candidates.peek().is_none() {
|
|
||||||
bail!("no release tag found in {url}");
|
|
||||||
}
|
|
||||||
|
|
||||||
for tag in candidates {
|
|
||||||
let release_url = format!("{}/repos/{repo}/releases/tags/{tag}", self.api);
|
|
||||||
if client.get(&release_url).send()?.status().is_success() {
|
|
||||||
return Ok(tag);
|
|
||||||
}
|
|
||||||
}
|
|
||||||
bail!("no release tag in {url} resolved to a real release via the API")
|
|
||||||
}
|
|
||||||
|
|
||||||
fn api(&self) -> &str {
|
|
||||||
&self.api
|
|
||||||
}
|
|
||||||
}
|
|
||||||
|
|
||||||
#[cfg(test)]
|
|
||||||
mod tests {
|
|
||||||
use super::*;
|
|
||||||
|
|
||||||
#[test]
|
|
||||||
fn default_points_at_real_github() {
|
|
||||||
let endpoints = GithubEndpoints::default();
|
|
||||||
assert_eq!(endpoints.web, "https://github.com");
|
|
||||||
assert_eq!(endpoints.api, "https://api.github.com");
|
|
||||||
}
|
|
||||||
|
|
||||||
fn atom_feed(tags: &[&str]) -> String {
|
|
||||||
let entries: String = tags
|
|
||||||
.iter()
|
|
||||||
.map(|t| {
|
|
||||||
format!(r#"<link rel="alternate" href="https://github.com/o/r/releases/tag/{t}"/>"#)
|
|
||||||
})
|
|
||||||
.collect();
|
|
||||||
format!("<feed>{entries}</feed>")
|
|
||||||
}
|
|
||||||
|
|
||||||
#[test]
|
|
||||||
fn latest_release_skips_tags_with_no_real_release() {
|
|
||||||
let mut server = mockito::Server::new();
|
|
||||||
let endpoints = GithubEndpoints {
|
|
||||||
web: server.url(),
|
|
||||||
api: server.url(),
|
|
||||||
};
|
|
||||||
|
|
||||||
// Mirrors the real scaleway-cli case: newest feed entry (a -dbg1
|
|
||||||
// tag) has no Release object behind it and 404s.
|
|
||||||
let _feed = server
|
|
||||||
.mock("GET", "/o/r/releases.atom")
|
|
||||||
.with_status(200)
|
|
||||||
.with_body(atom_feed(&["v2.62.0-dbg1", "v2.62.0"]))
|
|
||||||
.create();
|
|
||||||
let _missing = server
|
|
||||||
.mock("GET", "/repos/o/r/releases/tags/v2.62.0-dbg1")
|
|
||||||
.with_status(404)
|
|
||||||
.create();
|
|
||||||
let _real = server
|
|
||||||
.mock("GET", "/repos/o/r/releases/tags/v2.62.0")
|
|
||||||
.with_status(200)
|
|
||||||
.with_body("{}")
|
|
||||||
.create();
|
|
||||||
|
|
||||||
let client = reqwest::blocking::Client::new();
|
|
||||||
let tag = endpoints.latest_release(&client, "o/r").unwrap();
|
|
||||||
assert_eq!(tag, "v2.62.0");
|
|
||||||
}
|
|
||||||
|
|
||||||
#[test]
|
|
||||||
fn latest_release_errors_when_feed_has_no_tags() {
|
|
||||||
let mut server = mockito::Server::new();
|
|
||||||
let endpoints = GithubEndpoints {
|
|
||||||
web: server.url(),
|
|
||||||
api: server.url(),
|
|
||||||
};
|
|
||||||
let _feed = server
|
|
||||||
.mock("GET", "/o/r/releases.atom")
|
|
||||||
.with_status(200)
|
|
||||||
.with_body("<feed></feed>")
|
|
||||||
.create();
|
|
||||||
|
|
||||||
let client = reqwest::blocking::Client::new();
|
|
||||||
let err = endpoints.latest_release(&client, "o/r").unwrap_err();
|
|
||||||
assert!(err.to_string().contains("no release tag found"));
|
|
||||||
}
|
|
||||||
|
|
||||||
#[test]
|
|
||||||
fn latest_release_errors_when_no_candidate_resolves() {
|
|
||||||
let mut server = mockito::Server::new();
|
|
||||||
let endpoints = GithubEndpoints {
|
|
||||||
web: server.url(),
|
|
||||||
api: server.url(),
|
|
||||||
};
|
|
||||||
let _feed = server
|
|
||||||
.mock("GET", "/o/r/releases.atom")
|
|
||||||
.with_status(200)
|
|
||||||
.with_body(atom_feed(&["v1.0.0-dbg1"]))
|
|
||||||
.create();
|
|
||||||
let _missing = server
|
|
||||||
.mock("GET", "/repos/o/r/releases/tags/v1.0.0-dbg1")
|
|
||||||
.with_status(404)
|
|
||||||
.create();
|
|
||||||
|
|
||||||
let client = reqwest::blocking::Client::new();
|
|
||||||
let err = endpoints.latest_release(&client, "o/r").unwrap_err();
|
|
||||||
assert!(err.to_string().contains("resolved to a real release"));
|
|
||||||
}
|
|
||||||
|
|
||||||
#[test]
|
|
||||||
fn api_is_the_configured_api_root() {
|
|
||||||
let endpoints = GithubEndpoints {
|
|
||||||
web: "http://w".into(),
|
|
||||||
api: "http://a".into(),
|
|
||||||
};
|
|
||||||
assert_eq!(endpoints.api(), "http://a");
|
|
||||||
}
|
|
||||||
}
|
|
||||||
|
|
@ -1,57 +0,0 @@
|
||||||
//! Where a package's releases are published: the `ReleaseSource` trait,
|
|
||||||
//! one file per host implementing it, and `for_package`, which picks the
|
|
||||||
//! implementation for a package from its configured `source`. The
|
|
||||||
//! submodules are private and re-exported here, so the rest of the crate
|
|
||||||
//! imports everything from `crate::release_source` and never a host's
|
|
||||||
//! file.
|
|
||||||
|
|
||||||
mod contract;
|
|
||||||
mod forgejo;
|
|
||||||
mod github;
|
|
||||||
|
|
||||||
pub use contract::ReleaseSource;
|
|
||||||
pub use forgejo::ForgejoEndpoints;
|
|
||||||
pub use github::GithubEndpoints;
|
|
||||||
|
|
||||||
use crate::config::{Package, Source};
|
|
||||||
use anyhow::{Context, Result};
|
|
||||||
|
|
||||||
/// Defensive: errors only if a `forgejo-release` package has no
|
|
||||||
/// `base_url`, which `config::load_packages_dir` already guarantees.
|
|
||||||
pub fn for_package(pkg: &Package) -> Result<Box<dyn ReleaseSource>> {
|
|
||||||
match pkg.source {
|
|
||||||
Source::GithubRelease => Ok(Box::new(GithubEndpoints::default())),
|
|
||||||
Source::ForgejoRelease => {
|
|
||||||
let base_url = pkg
|
|
||||||
.base_url
|
|
||||||
.as_deref()
|
|
||||||
.context("source = \"forgejo-release\" needs a base_url")?;
|
|
||||||
Ok(Box::new(ForgejoEndpoints::from_base_url(base_url)))
|
|
||||||
}
|
|
||||||
}
|
|
||||||
}
|
|
||||||
|
|
||||||
#[cfg(test)]
|
|
||||||
mod tests {
|
|
||||||
use super::*;
|
|
||||||
use crate::test_support::same_origin_package as package;
|
|
||||||
|
|
||||||
#[test]
|
|
||||||
fn github_source_uses_real_github() {
|
|
||||||
let source = for_package(&package("")).unwrap();
|
|
||||||
assert_eq!(source.api(), "https://api.github.com");
|
|
||||||
}
|
|
||||||
|
|
||||||
#[test]
|
|
||||||
fn forgejo_source_uses_the_instances_api() {
|
|
||||||
let pkg = package("source = \"forgejo-release\"\nbase_url = \"https://code.example.com\"");
|
|
||||||
let source = for_package(&pkg).unwrap();
|
|
||||||
assert_eq!(source.api(), "https://code.example.com/api/v1");
|
|
||||||
}
|
|
||||||
|
|
||||||
#[test]
|
|
||||||
fn forgejo_source_without_base_url_errors() {
|
|
||||||
let err = for_package(&package("source = \"forgejo-release\"")).unwrap_err();
|
|
||||||
assert!(err.to_string().contains("needs a base_url"));
|
|
||||||
}
|
|
||||||
}
|
|
||||||
|
|
@ -1,6 +1,6 @@
|
||||||
//! Persists two independent per-package facts as plain files: the last
|
//! Persists two independent per-package facts as plain files: the last
|
||||||
//! published version, and any version currently pending human review.
|
//! published version, and any version currently pending human review.
|
||||||
//! The only module that touches the state directory (see `paths.rs`) on disk.
|
//! The only module that touches `state/` on disk.
|
||||||
|
|
||||||
use anyhow::Result;
|
use anyhow::Result;
|
||||||
use std::path::Path;
|
use std::path::Path;
|
||||||
|
|
|
||||||
|
|
@ -1,74 +1,21 @@
|
||||||
//! Test-only fixture helpers shared across modules' `#[cfg(test)]` code
|
//! Test-only fixture helpers shared across modules' `#[cfg(test)]` code
|
||||||
//! — not production code, and not built outside `cargo test`. See
|
//! (`publisher`, `sanity`) — not production code, and not built outside
|
||||||
//! docs/ARCHITECTURE.md > "organize by pipeline stage, not by layer": this
|
//! `cargo test`. See docs/ARCHITECTURE.md > "organize by pipeline stage, not
|
||||||
//! exists to remove specific pieces of duplication (near-identical copies
|
//! by layer": this exists to remove one specific piece of duplication
|
||||||
//! of "write an executable shell script" and of "build a same-origin
|
//! (two near-identical copies of "write an executable shell script"), not
|
||||||
//! `Package` from TOML"), not as a general test-utils dump.
|
//! as a general test-utils dump.
|
||||||
|
|
||||||
use crate::config::Package;
|
|
||||||
use std::path::{Path, PathBuf};
|
use std::path::{Path, PathBuf};
|
||||||
use std::process::Command;
|
|
||||||
use std::time::{Duration, Instant};
|
|
||||||
|
|
||||||
/// Set by `wait_until_executable`'s probe so the script exits before
|
|
||||||
/// running its real body.
|
|
||||||
const PROBE_ENV: &str = "PKGWATCH_TEST_SCRIPT_PROBE";
|
|
||||||
/// `ETXTBSY`: exec of a file some process still has open for writing.
|
|
||||||
const TEXT_FILE_BUSY: i32 = 26;
|
|
||||||
|
|
||||||
/// Writes an executable `#!/bin/sh` script named `name` into `dir`,
|
/// Writes an executable `#!/bin/sh` script named `name` into `dir`,
|
||||||
/// running `body` as its contents. Used to stand in for a real binary
|
/// running `body` as its contents. Used to stand in for a real binary
|
||||||
/// (`repo-add`, a package's own `--version` command) in tests, without
|
/// (`repo-add`, a package's own `--version` command) in tests, without
|
||||||
/// needing the real tool installed or a mutated global `PATH`.
|
/// needing the real tool installed or a mutated global `PATH`.
|
||||||
///
|
|
||||||
/// Doesn't return until the script can actually be exec'd. `cargo test`
|
|
||||||
/// runs tests on parallel threads, and a `fork` on another thread between
|
|
||||||
/// this function's write-open and close leaves the child holding a copy of
|
|
||||||
/// the write fd until it execs, so an immediate exec of the new script can
|
|
||||||
/// fail with `Text file busy`. Once no holder is left none can appear (our
|
|
||||||
/// fd is closed), so a probe exec that succeeds proves later ones will.
|
|
||||||
pub(crate) fn write_executable_script(dir: &Path, name: &str, body: &str) -> PathBuf {
|
pub(crate) fn write_executable_script(dir: &Path, name: &str, body: &str) -> PathBuf {
|
||||||
let path = dir.join(name);
|
let path = dir.join(name);
|
||||||
std::fs::write(
|
std::fs::write(&path, format!("#!/bin/sh\n{body}\n")).unwrap();
|
||||||
&path,
|
|
||||||
format!("#!/bin/sh\n[ -z \"${PROBE_ENV}\" ] || exit 0\n{body}\n"),
|
|
||||||
)
|
|
||||||
.unwrap();
|
|
||||||
let mut perms = std::fs::metadata(&path).unwrap().permissions();
|
let mut perms = std::fs::metadata(&path).unwrap().permissions();
|
||||||
std::os::unix::fs::PermissionsExt::set_mode(&mut perms, 0o755);
|
std::os::unix::fs::PermissionsExt::set_mode(&mut perms, 0o755);
|
||||||
std::fs::set_permissions(&path, perms).unwrap();
|
std::fs::set_permissions(&path, perms).unwrap();
|
||||||
wait_until_executable(&path);
|
|
||||||
path
|
path
|
||||||
}
|
}
|
||||||
|
|
||||||
fn wait_until_executable(path: &Path) {
|
|
||||||
let deadline = Instant::now() + Duration::from_secs(5);
|
|
||||||
loop {
|
|
||||||
match Command::new(path).env(PROBE_ENV, "1").status() {
|
|
||||||
Ok(_) => return,
|
|
||||||
Err(err) if err.raw_os_error() == Some(TEXT_FILE_BUSY) && Instant::now() < deadline => {
|
|
||||||
std::thread::sleep(Duration::from_millis(5));
|
|
||||||
}
|
|
||||||
Err(err) => panic!("probe-exec of {} failed: {err}", path.display()),
|
|
||||||
}
|
|
||||||
}
|
|
||||||
}
|
|
||||||
|
|
||||||
/// A `same-origin-sha256` `Package` for repo `o/r`, asset `thing.tar.gz`,
|
|
||||||
/// checksum asset `SHA256SUMS`. `extra` is spliced in as top-level keys
|
|
||||||
/// before the `[verification]` table (e.g. `source`/`base_url`), and is
|
|
||||||
/// parsed directly rather than through `config::load_packages_dir`, so it
|
|
||||||
/// skips load-time validation.
|
|
||||||
pub(crate) fn same_origin_package(extra: &str) -> Package {
|
|
||||||
toml::from_str(&format!(
|
|
||||||
r#"
|
|
||||||
repo = "o/r"
|
|
||||||
asset_pattern = "thing.tar.gz"
|
|
||||||
{extra}
|
|
||||||
[verification]
|
|
||||||
method = "same-origin-sha256"
|
|
||||||
checksum_asset_pattern = "SHA256SUMS"
|
|
||||||
"#
|
|
||||||
))
|
|
||||||
.unwrap()
|
|
||||||
}
|
|
||||||
|
|
|
||||||
|
|
@ -6,6 +6,7 @@
|
||||||
use crate::checker::version_from_tag;
|
use crate::checker::version_from_tag;
|
||||||
use crate::config::Verification;
|
use crate::config::Verification;
|
||||||
use crate::fetcher;
|
use crate::fetcher;
|
||||||
|
use crate::github::GithubEndpoints;
|
||||||
use crate::hash;
|
use crate::hash;
|
||||||
use anyhow::{Context, Result, bail};
|
use anyhow::{Context, Result, bail};
|
||||||
use std::path::Path;
|
use std::path::Path;
|
||||||
|
|
@ -22,7 +23,7 @@ pub struct VerificationResult {
|
||||||
/// each tier does and does not prove.
|
/// each tier does and does not prove.
|
||||||
pub fn verify(
|
pub fn verify(
|
||||||
client: &reqwest::blocking::Client,
|
client: &reqwest::blocking::Client,
|
||||||
api: &str,
|
endpoints: &GithubEndpoints,
|
||||||
verification: &Verification,
|
verification: &Verification,
|
||||||
repo: &str,
|
repo: &str,
|
||||||
tag: &str,
|
tag: &str,
|
||||||
|
|
@ -35,8 +36,14 @@ pub fn verify(
|
||||||
} => {
|
} => {
|
||||||
let checksum_asset_name =
|
let checksum_asset_name =
|
||||||
checksum_asset_pattern.replace("{version}", version_from_tag(tag));
|
checksum_asset_pattern.replace("{version}", version_from_tag(tag));
|
||||||
let checksum_asset =
|
let checksum_asset = fetcher::download_asset(
|
||||||
fetcher::download_asset(client, api, repo, tag, &checksum_asset_name, dest_dir)?;
|
client,
|
||||||
|
endpoints,
|
||||||
|
repo,
|
||||||
|
tag,
|
||||||
|
&checksum_asset_name,
|
||||||
|
dest_dir,
|
||||||
|
)?;
|
||||||
let checksum_text = std::fs::read_to_string(&checksum_asset.path)?;
|
let checksum_text = std::fs::read_to_string(&checksum_asset.path)?;
|
||||||
let artifact_name = artifact_path
|
let artifact_name = artifact_path
|
||||||
.file_name()
|
.file_name()
|
||||||
|
|
@ -177,7 +184,10 @@ mod tests {
|
||||||
#[test]
|
#[test]
|
||||||
fn verify_same_origin_sha256_passes_on_matching_checksum() {
|
fn verify_same_origin_sha256_passes_on_matching_checksum() {
|
||||||
let mut server = mockito::Server::new();
|
let mut server = mockito::Server::new();
|
||||||
let api = server.url();
|
let endpoints = GithubEndpoints {
|
||||||
|
web: server.url(),
|
||||||
|
api: server.url(),
|
||||||
|
};
|
||||||
let dest_dir = tempfile::tempdir().unwrap();
|
let dest_dir = tempfile::tempdir().unwrap();
|
||||||
let artifact_path = dest_dir.path().join("thing.tar.gz");
|
let artifact_path = dest_dir.path().join("thing.tar.gz");
|
||||||
std::fs::write(&artifact_path, b"hello world").unwrap();
|
std::fs::write(&artifact_path, b"hello world").unwrap();
|
||||||
|
|
@ -204,7 +214,7 @@ mod tests {
|
||||||
let client = reqwest::blocking::Client::new();
|
let client = reqwest::blocking::Client::new();
|
||||||
let result = verify(
|
let result = verify(
|
||||||
&client,
|
&client,
|
||||||
&api,
|
&endpoints,
|
||||||
&verification,
|
&verification,
|
||||||
"o/r",
|
"o/r",
|
||||||
"v1.0.0",
|
"v1.0.0",
|
||||||
|
|
@ -220,7 +230,10 @@ mod tests {
|
||||||
#[test]
|
#[test]
|
||||||
fn verify_same_origin_sha256_fails_on_mismatched_checksum() {
|
fn verify_same_origin_sha256_fails_on_mismatched_checksum() {
|
||||||
let mut server = mockito::Server::new();
|
let mut server = mockito::Server::new();
|
||||||
let api = server.url();
|
let endpoints = GithubEndpoints {
|
||||||
|
web: server.url(),
|
||||||
|
api: server.url(),
|
||||||
|
};
|
||||||
let dest_dir = tempfile::tempdir().unwrap();
|
let dest_dir = tempfile::tempdir().unwrap();
|
||||||
let artifact_path = dest_dir.path().join("thing.tar.gz");
|
let artifact_path = dest_dir.path().join("thing.tar.gz");
|
||||||
std::fs::write(&artifact_path, b"hello world").unwrap();
|
std::fs::write(&artifact_path, b"hello world").unwrap();
|
||||||
|
|
@ -248,7 +261,7 @@ mod tests {
|
||||||
let client = reqwest::blocking::Client::new();
|
let client = reqwest::blocking::Client::new();
|
||||||
let result = verify(
|
let result = verify(
|
||||||
&client,
|
&client,
|
||||||
&api,
|
&endpoints,
|
||||||
&verification,
|
&verification,
|
||||||
"o/r",
|
"o/r",
|
||||||
"v1.0.0",
|
"v1.0.0",
|
||||||
|
|
|
||||||
|
|
@ -1,11 +0,0 @@
|
||||||
# Triggered by pkgwatch.service's OnFailure=. Covers what the in-process
|
|
||||||
# notifier can't: a verification failure, build failure, or network error
|
|
||||||
# exits non-zero, and that would otherwise only show up in the journal.
|
|
||||||
[Unit]
|
|
||||||
Description=Notify that a pkgwatch run failed
|
|
||||||
|
|
||||||
[Service]
|
|
||||||
Type=oneshot
|
|
||||||
# Absolute path: systemd requires one, unlike the in-process notifier,
|
|
||||||
# which resolves notify-send from PATH.
|
|
||||||
ExecStart=/usr/bin/notify-send --app-name=pkgwatch --urgency=critical "pkgwatch run failed" "See: journalctl --user -u pkgwatch.service"
|
|
||||||
|
|
@ -1,16 +0,0 @@
|
||||||
# User-level oneshot: one check -> fetch -> verify -> build -> publish pass.
|
|
||||||
# Install: see docs/SPEC.md > Scheduling.
|
|
||||||
#
|
|
||||||
# No WorkingDirectory: config, state and work dirs come from the XDG paths
|
|
||||||
# in src/paths.rs (see docs/SPEC.md > Paths), not the cwd.
|
|
||||||
# The binary is the release build in the main checkout (`cargo build
|
|
||||||
# --release`), so a rebuild is what picks up code changes.
|
|
||||||
[Unit]
|
|
||||||
Description=pkgwatch: check tracked packages for new upstream releases
|
|
||||||
OnFailure=pkgwatch-failure.service
|
|
||||||
|
|
||||||
[Service]
|
|
||||||
Type=oneshot
|
|
||||||
ExecStart=%h/dev/pkgwatch/target/release/pkgwatch
|
|
||||||
# Builds (makepkg, large Go/Rust binaries) can legitimately take a while.
|
|
||||||
TimeoutStartSec=30min
|
|
||||||
|
|
@ -1,18 +0,0 @@
|
||||||
# Periodic, not persistent (see docs/SPEC.md > Vision): no catch-up burst
|
|
||||||
# for missed ticks. The laptop is only on while logged in, so the user
|
|
||||||
# manager starting (= login) is what matters: OnStartupSec runs a check
|
|
||||||
# right after login (10s, so the session bus and network are up) instead
|
|
||||||
# of waiting up to an hour for the next tick.
|
|
||||||
#
|
|
||||||
# No RandomizedDelaySec: it applies to every trigger, including the
|
|
||||||
# startup one, and a single machine has no herd to spread out anyway.
|
|
||||||
[Unit]
|
|
||||||
Description=Run pkgwatch at login and hourly
|
|
||||||
|
|
||||||
[Timer]
|
|
||||||
OnStartupSec=10s
|
|
||||||
OnCalendar=hourly
|
|
||||||
Persistent=false
|
|
||||||
|
|
||||||
[Install]
|
|
||||||
WantedBy=timers.target
|
|
||||||
Loading…
Reference in a new issue