schaefera/pkgwatch
1
0
Fork 0
Code Issues Pull requests Projects Releases Packages Wiki Activity Actions

Add a Forgejo release source #5

Merged
schaefera merged 5 commits from worktree-forgejo-source into master 2026-09-20 09:20:00 +00:00
Conversation 0 Commits 5 Files changed 14 +336 -316
9 changed files with 336 additions and 316 deletions
Show stats Download patch file Download diff file Expand all files Collapse all files
Showing only changes of commit ba2c5c2d02 - Show all commits

9
docs/ARCHITECTURE.md
View file

@ -61,8 +61,9 @@ practice rather than asserted from habit — see Further reading.
Same testability goal as #3, applied to the specific ways this program Same testability goal as #3, applied to the specific ways this program
reaches outside itself. A consistent shape beats ad hoc mocking invented reaches outside itself. A consistent shape beats ad hoc mocking invented
per call site. per call site.
**Already in force**: `source::Endpoints` (checker), the plain API root **Already in force**: `GithubEndpoints`/`ForgejoEndpoints` (the
it hands to fetcher/verifier, `repo_add_bin` and the pacman.conf path (publisher), `PKGWATCH_REPO_DIR` `ReleaseSource` implementations, whose API root is what fetcher/verifier
take), `repo_add_bin` and the pacman.conf path (publisher), `PKGWATCH_REPO_DIR`
(main, for manual dry runs against a scratch repo instead of the real (main, for manual dry runs against a scratch repo instead of the real
one). A new external call follows the same shape: production code calls one). A new external call follows the same shape: production code calls
a thin wrapper with the real default; tests call the parameterized a thin wrapper with the real default; tests call the parameterized
@ -106,8 +107,8 @@ practice rather than asserted from habit — see Further reading.
directory, but the same information — why this way and not the obvious directory, but the same information — why this way and not the obvious
alternative — needs to live somewhere a future reader will actually see alternative — needs to live somewhere a future reader will actually see
it: the doc comment on the thing itself. it: the doc comment on the thing itself.
**Example already here**: `checker.rs`'s doc comment on **Example already here**: `github.rs`'s doc comment on
`latest_github_release` explains why the newest Atom-feed entry isn't `GithubEndpoints::latest_release` explains why the newest Atom-feed entry isn't
trusted outright (scaleway-cli's `-dbg1` tag has no real Release behind trusted outright (scaleway-cli's `-dbg1` tag has no real Release behind
it) — the reasoning lives right next to the code it justifies, not in a it) — the reasoning lives right next to the code it justifies, not in a
commit message or a separate design doc no one will find later. commit message or a separate design doc no one will find later.

19
docs/SPEC.md
View file

@ -287,7 +287,7 @@ implements `repo`, `asset_pattern`, and `verification.method`
matches the line by filename instead of assuming a single-hash file. matches the line by filename instead of assuming a single-hash file.
- Separately, scaleway-cli's Atom feed lists a `vX.Y.Z-dbg1` tag newest, - Separately, scaleway-cli's Atom feed lists a `vX.Y.Z-dbg1` tag newest,
with no real Release object behind it (`releases/tags/<tag>` 404s) — with no real Release object behind it (`releases/tags/<tag>` 404s) —
`checker::latest_github_release` now confirms each feed candidate the GitHub source's `latest_release` now confirms each feed candidate
against the releases API in feed order rather than trusting the first against the releases API in feed order rather than trusting the first
entry outright. entry outright.
@ -373,10 +373,15 @@ Open questions on the schema:
likely reuses `nvchecker`'s logic/sources conceptually for non-GitHub likely reuses `nvchecker`'s logic/sources conceptually for non-GitHub
sources eventually. For GitHub sources, prefers the `github-atom` feed sources eventually. For GitHub sources, prefers the `github-atom` feed
(see Scaling > Check method) over unconditional REST polling. (see Scaling > Check method) over unconditional REST polling.
*(Implemented for GitHub and Forgejo — `src/checker.rs`. GitHub *(Implemented for GitHub and Forgejo. `src/checker.rs` defines the
regex-matches the first `releases/tag/<tag>` link in the feed rather than `ReleaseSource` trait (latest release + API root); each host implements
doing a full XML parse; fine while the feed's newest-entry-first shape it in its own module (`github.rs`, `forgejo.rs`), and `source.rs` picks
holds, revisit if that ever changes. Forgejo is one call to one per package, so adding a host doesn't touch existing ones. A trait
rather than an enum match because there are now two real hosts with
genuinely different logic. GitHub regex-matches the first
`releases/tag/<tag>` link in the feed rather than doing a full XML
parse; fine while the feed's newest-entry-first shape holds, revisit if
that ever changes. Forgejo is one call to
`<base_url>/api/v1/repos/<repo>/releases/latest`, which already returns `<base_url>/api/v1/repos/<repo>/releases/latest`, which already returns
only the newest non-draft, non-prerelease release, so it needs none of only the newest non-draft, non-prerelease release, so it needs none of
GitHub's confirm-each-tag step. `check_interval`/per-package cadence not GitHub's confirm-each-tag step. `check_interval`/per-package cadence not
@ -519,8 +524,8 @@ Open questions on the schema:
confirmed correct — no build-provenance attestations upstream. confirmed correct — no build-provenance attestations upstream.
Required adding `{version}`-placeholder support to `asset_pattern`/ Required adding `{version}`-placeholder support to `asset_pattern`/
`checksum_asset_pattern`, filename-matched parsing of combined `checksum_asset_pattern`, filename-matched parsing of combined
multi-asset checksum files, and having `latest_github_release` multi-asset checksum files, and having the GitHub source's
confirm each Atom-feed candidate against the releases API (this `latest_release` confirm each Atom-feed candidate against the releases API (this
repo's newest feed entry, a `-dbg1` tag, has no real Release behind repo's newest feed entry, a `-dbg1` tag, has no real Release behind
it). Still just flags for human review, same as any tier 4-6 pass — it). Still just flags for human review, same as any tier 4-6 pass —
not auto-installed; see the unchecked build/publish item below. not auto-installed; see the unchecked build/publish item below.

250
src/checker.rs
View file

@ -1,89 +1,24 @@
use crate::github::GithubEndpoints; //! The check stage's contract: a `ReleaseSource` says what a package's
use crate::source::Endpoints; //! latest release is and where its releases API lives, and
use anyhow::{Context, Result, bail}; //! `version_from_tag` turns the resulting tag into a version string. Each
use regex::Regex; //! hosting service implements the trait in its own module (`github`,
use serde::Deserialize; //! `forgejo`) so per-host logic doesn't accumulate here;
//! `source::for_package` picks the implementation for a package.
/// Resolves the latest release tag for `repo` on whichever service use anyhow::Result;
/// `endpoints` points at.
pub fn latest_release(
client: &reqwest::blocking::Client,
endpoints: &Endpoints,
repo: &str,
) -> Result<String> {
match endpoints {
Endpoints::Github(github) => latest_github_release(client, github, repo),
Endpoints::Forgejo { api } => latest_forgejo_release(client, api, repo),
}
}
/// Resolves the latest release tag for `repo` on a Forgejo/Gitea instance. /// A hosting service a package's releases are published on. `Debug` so a
/// /// `Box<dyn ReleaseSource>` can sit in a `Result` that tests unwrap.
/// One call, unlike GitHub's feed-then-confirm dance below: Forgejo's pub trait ReleaseSource: std::fmt::Debug {
/// `releases/latest` already returns only the newest non-draft, /// The latest release tag for `repo`.
/// non-prerelease *release object*, so a stray tag with no release behind fn latest_release(&self, client: &reqwest::blocking::Client, repo: &str) -> Result<String>;
/// it (GitHub's scaleway-cli `-dbg1` problem) can't be returned.
pub fn latest_forgejo_release(
client: &reqwest::blocking::Client,
api: &str,
repo: &str,
) -> Result<String> {
#[derive(Deserialize)]
struct Latest {
tag_name: String,
}
let url = format!("{api}/repos/{repo}/releases/latest"); /// The releases API root, which `fetcher` and `verifier` build their
let response = client.get(&url).send()?; /// own paths under. GitHub and Forgejo both serve
// Forgejo answers 404 both for an unknown repo and for one with no /// `/repos/{owner}/{repo}/releases/tags/{tag}` there with the same
// releases yet — the common state for a project's very first release. /// `assets[].{name, browser_download_url}` shape, which is why those
if response.status() == reqwest::StatusCode::NOT_FOUND { /// stages need only this and not the source itself.
bail!("no published release found at {url} (repo missing, or nothing released yet)"); fn api(&self) -> &str;
}
let latest: Latest = response
.error_for_status()
.with_context(|| format!("fetching latest release from {url}"))?
.json()?;
Ok(latest.tag_name)
}
/// Resolves the latest release tag for `repo` via its public Atom feed.
///
/// Deliberately not a full XML parse: the feed lists entries newest-first,
/// and each `<link rel="alternate" .../releases/tag/<tag>"/>` is matched
/// in document order. Revisit with a real XML parser if GitHub's feed
/// shape ever changes.
///
/// The feed can list a tag newer than any tag with a real Release object
/// behind it — observed on scaleway/scaleway-cli, which pushes a
/// `vX.Y.Z-dbg1` tag (no corresponding Release; `releases/tags/<tag>`
/// 404s) right after each real release, and that tag sorts newest in the
/// feed. So each candidate is confirmed against the releases API in feed
/// order, returning the first that actually resolves.
pub fn latest_github_release(
client: &reqwest::blocking::Client,
endpoints: &GithubEndpoints,
repo: &str,
) -> Result<String> {
let url = format!("{}/{repo}/releases.atom", endpoints.web);
let body = client.get(&url).send()?.error_for_status()?.text()?;
let re = Regex::new(r#"releases/tag/([^"]+)""#)?;
let mut candidates = re
.captures_iter(&body)
.map(|caps| caps[1].to_string())
.peekable();
if candidates.peek().is_none() {
bail!("no release tag found in {url}");
}
for tag in candidates {
let release_url = format!("{}/repos/{repo}/releases/tags/{tag}", endpoints.api);
if client.get(&release_url).send()?.status().is_success() {
return Ok(tag);
}
}
bail!("no release tag in {url} resolved to a real release via the API")
} }
/// Strips a leading `v` from a release tag, e.g. `v2.62.0` -> `2.62.0`. /// Strips a leading `v` from a release tag, e.g. `v2.62.0` -> `2.62.0`.
@ -110,153 +45,4 @@ mod tests {
fn version_from_tag_leaves_bare_version_unchanged() { fn version_from_tag_leaves_bare_version_unchanged() {
assert_eq!(version_from_tag("0.12.15"), "0.12.15"); assert_eq!(version_from_tag("0.12.15"), "0.12.15");
} }
fn atom_feed(tags: &[&str]) -> String {
let entries: String = tags
.iter()
.map(|t| {
format!(r#"<link rel="alternate" href="https://github.com/o/r/releases/tag/{t}"/>"#)
})
.collect();
format!("<feed>{entries}</feed>")
}
#[test]
fn latest_github_release_skips_tags_with_no_real_release() {
let mut server = mockito::Server::new();
let endpoints = GithubEndpoints {
web: server.url(),
api: server.url(),
};
// Mirrors the real scaleway-cli case: newest feed entry (a -dbg1
// tag) has no Release object behind it and 404s.
let _feed = server
.mock("GET", "/o/r/releases.atom")
.with_status(200)
.with_body(atom_feed(&["v2.62.0-dbg1", "v2.62.0"]))
.create();
let _missing = server
.mock("GET", "/repos/o/r/releases/tags/v2.62.0-dbg1")
.with_status(404)
.create();
let _real = server
.mock("GET", "/repos/o/r/releases/tags/v2.62.0")
.with_status(200)
.with_body("{}")
.create();
let client = reqwest::blocking::Client::new();
let tag = latest_github_release(&client, &endpoints, "o/r").unwrap();
assert_eq!(tag, "v2.62.0");
}
#[test]
fn latest_github_release_errors_when_feed_has_no_tags() {
let mut server = mockito::Server::new();
let endpoints = GithubEndpoints {
web: server.url(),
api: server.url(),
};
let _feed = server
.mock("GET", "/o/r/releases.atom")
.with_status(200)
.with_body("<feed></feed>")
.create();
let client = reqwest::blocking::Client::new();
let err = latest_github_release(&client, &endpoints, "o/r").unwrap_err();
assert!(err.to_string().contains("no release tag found"));
}
#[test]
fn latest_github_release_errors_when_no_candidate_resolves() {
let mut server = mockito::Server::new();
let endpoints = GithubEndpoints {
web: server.url(),
api: server.url(),
};
let _feed = server
.mock("GET", "/o/r/releases.atom")
.with_status(200)
.with_body(atom_feed(&["v1.0.0-dbg1"]))
.create();
let _missing = server
.mock("GET", "/repos/o/r/releases/tags/v1.0.0-dbg1")
.with_status(404)
.create();
let client = reqwest::blocking::Client::new();
let err = latest_github_release(&client, &endpoints, "o/r").unwrap_err();
assert!(err.to_string().contains("resolved to a real release"));
}
#[test]
fn latest_forgejo_release_returns_tag_name() {
let mut server = mockito::Server::new();
let _latest = server
.mock("GET", "/repos/o/r/releases/latest")
.with_status(200)
.with_body(r#"{"tag_name": "v0.1.0", "assets": []}"#)
.create();
let client = reqwest::blocking::Client::new();
let tag = latest_forgejo_release(&client, &server.url(), "o/r").unwrap();
assert_eq!(tag, "v0.1.0");
}
#[test]
fn latest_forgejo_release_names_the_no_releases_case() {
let mut server = mockito::Server::new();
let _latest = server
.mock("GET", "/repos/o/r/releases/latest")
.with_status(404)
.create();
let client = reqwest::blocking::Client::new();
let err = latest_forgejo_release(&client, &server.url(), "o/r").unwrap_err();
assert!(err.to_string().contains("no published release"));
}
#[test]
fn latest_forgejo_release_surfaces_server_errors() {
let mut server = mockito::Server::new();
let _latest = server
.mock("GET", "/repos/o/r/releases/latest")
.with_status(500)
.create();
let client = reqwest::blocking::Client::new();
let err = latest_forgejo_release(&client, &server.url(), "o/r").unwrap_err();
assert!(err.to_string().contains("fetching latest release"));
}
#[test]
fn latest_release_dispatches_on_endpoint_kind() {
let mut server = mockito::Server::new();
let _forgejo = server
.mock("GET", "/repos/o/r/releases/latest")
.with_status(200)
.with_body(r#"{"tag_name": "v2.0.0"}"#)
.create();
let _feed = server
.mock("GET", "/o/r/releases.atom")
.with_body(atom_feed(&["v1.0.0"]))
.create();
let _release = server
.mock("GET", "/repos/o/r/releases/tags/v1.0.0")
.with_status(200)
.with_body("{}")
.create();
let client = reqwest::blocking::Client::new();
let forgejo = Endpoints::Forgejo { api: server.url() };
assert_eq!(latest_release(&client, &forgejo, "o/r").unwrap(), "v2.0.0");
let github = Endpoints::Github(GithubEndpoints {
web: server.url(),
api: server.url(),
});
assert_eq!(latest_release(&client, &github, "o/r").unwrap(), "v1.0.0");
}
} }

2
src/fetcher.rs
View file

@ -29,7 +29,7 @@ pub struct DownloadedAsset {
/// Downloads the release asset named exactly `asset_name` for `repo`@`tag` /// Downloads the release asset named exactly `asset_name` for `repo`@`tag`
/// into `dest_dir`, returning the local path and its origin URL. `api` is /// into `dest_dir`, returning the local path and its origin URL. `api` is
/// the releases API root (see `source::Endpoints::api`); GitHub and Forgejo /// the releases API root (see `ReleaseSource::api`); GitHub and Forgejo
/// serve the same endpoint and JSON shape under it. /// serve the same endpoint and JSON shape under it.
pub fn download_asset( pub fn download_asset(
client: &reqwest::blocking::Client, client: &reqwest::blocking::Client,

112
src/forgejo.rs Normal file
View file

@ -0,0 +1,112 @@
//! A Forgejo (or Gitea) instance as a release source: its API root, and
//! how to find a repo's latest release there.
use crate::checker::ReleaseSource;
use anyhow::{Context, Result, bail};
use serde::Deserialize;
#[derive(Debug, Clone)]
pub struct ForgejoEndpoints {
/// The instance's API root, i.e. `<base_url>/api/v1`.
pub api: String,
}
impl ForgejoEndpoints {
/// `base_url` is the instance's web root, e.g.
/// `https://code.austinschaefer.com`; a trailing slash is tolerated.
pub fn from_base_url(base_url: &str) -> Self {
Self {
api: format!("{}/api/v1", base_url.trim_end_matches('/')),
}
}
}
impl ReleaseSource for ForgejoEndpoints {
/// One call, unlike GitHub's feed-then-confirm dance: Forgejo's
/// `releases/latest` already returns only the newest non-draft,
/// non-prerelease *release object*, so a stray tag with no release
/// behind it (GitHub's scaleway-cli `-dbg1` problem) can't be returned.
fn latest_release(&self, client: &reqwest::blocking::Client, repo: &str) -> Result<String> {
#[derive(Deserialize)]
struct Latest {
tag_name: String,
}
let url = format!("{}/repos/{repo}/releases/latest", self.api);
let response = client.get(&url).send()?;
// Forgejo answers 404 both for an unknown repo and for one with no
// releases yet — the common state for a project's very first
// release.
if response.status() == reqwest::StatusCode::NOT_FOUND {
bail!("no published release found at {url} (repo missing, or nothing released yet)");
}
let latest: Latest = response
.error_for_status()
.with_context(|| format!("fetching latest release from {url}"))?
.json()?;
Ok(latest.tag_name)
}
fn api(&self) -> &str {
&self.api
}
}
#[cfg(test)]
mod tests {
use super::*;
#[test]
fn from_base_url_appends_api_v1() {
let endpoints = ForgejoEndpoints::from_base_url("https://code.example.com");
assert_eq!(endpoints.api(), "https://code.example.com/api/v1");
}
#[test]
fn from_base_url_tolerates_trailing_slash() {
let endpoints = ForgejoEndpoints::from_base_url("https://code.example.com/");
assert_eq!(endpoints.api(), "https://code.example.com/api/v1");
}
#[test]
fn latest_release_returns_tag_name() {
let mut server = mockito::Server::new();
let _latest = server
.mock("GET", "/repos/o/r/releases/latest")
.with_status(200)
.with_body(r#"{"tag_name": "v0.1.0", "assets": []}"#)
.create();
let client = reqwest::blocking::Client::new();
let endpoints = ForgejoEndpoints { api: server.url() };
assert_eq!(endpoints.latest_release(&client, "o/r").unwrap(), "v0.1.0");
}
#[test]
fn latest_release_names_the_no_releases_case() {
let mut server = mockito::Server::new();
let _latest = server
.mock("GET", "/repos/o/r/releases/latest")
.with_status(404)
.create();
let client = reqwest::blocking::Client::new();
let endpoints = ForgejoEndpoints { api: server.url() };
let err = endpoints.latest_release(&client, "o/r").unwrap_err();
assert!(err.to_string().contains("no published release"));
}
#[test]
fn latest_release_surfaces_server_errors() {
let mut server = mockito::Server::new();
let _latest = server
.mock("GET", "/repos/o/r/releases/latest")
.with_status(500)
.create();
let client = reqwest::blocking::Client::new();
let endpoints = ForgejoEndpoints { api: server.url() };
let err = endpoints.latest_release(&client, "o/r").unwrap_err();
assert!(err.to_string().contains("fetching latest release"));
}
}

137
src/github.rs
View file

@ -1,3 +1,10 @@
//! GitHub as a release source: its endpoints, and how to find a repo's
//! latest release there.
use crate::checker::ReleaseSource;
use anyhow::{Result, bail};
use regex::Regex;
/// Base URLs for GitHub's public web host (Atom feeds, release pages) and /// Base URLs for GitHub's public web host (Atom feeds, release pages) and
/// its REST API, factored out so tests can point both at a local mock /// its REST API, factored out so tests can point both at a local mock
/// server instead of the real github.com/api.github.com. /// server instead of the real github.com/api.github.com.
@ -16,6 +23,47 @@ impl Default for GithubEndpoints {
} }
} }
impl ReleaseSource for GithubEndpoints {
/// Resolves the latest release tag for `repo` via its public Atom feed.
///
/// Deliberately not a full XML parse: the feed lists entries
/// newest-first, and each `<link rel="alternate"
/// .../releases/tag/<tag>"/>` is matched in document order. Revisit
/// with a real XML parser if GitHub's feed shape ever changes.
///
/// The feed can list a tag newer than any tag with a real Release
/// object behind it — observed on scaleway/scaleway-cli, which pushes a
/// `vX.Y.Z-dbg1` tag (no corresponding Release; `releases/tags/<tag>`
/// 404s) right after each real release, and that tag sorts newest in
/// the feed. So each candidate is confirmed against the releases API in
/// feed order, returning the first that actually resolves.
fn latest_release(&self, client: &reqwest::blocking::Client, repo: &str) -> Result<String> {
let url = format!("{}/{repo}/releases.atom", self.web);
let body = client.get(&url).send()?.error_for_status()?.text()?;
let re = Regex::new(r#"releases/tag/([^"]+)""#)?;
let mut candidates = re
.captures_iter(&body)
.map(|caps| caps[1].to_string())
.peekable();
if candidates.peek().is_none() {
bail!("no release tag found in {url}");
}
for tag in candidates {
let release_url = format!("{}/repos/{repo}/releases/tags/{tag}", self.api);
if client.get(&release_url).send()?.status().is_success() {
return Ok(tag);
}
}
bail!("no release tag in {url} resolved to a real release via the API")
}
fn api(&self) -> &str {
&self.api
}
}
#[cfg(test)] #[cfg(test)]
mod tests { mod tests {
use super::*; use super::*;
@ -26,4 +74,93 @@ mod tests {
assert_eq!(endpoints.web, "https://github.com"); assert_eq!(endpoints.web, "https://github.com");
assert_eq!(endpoints.api, "https://api.github.com"); assert_eq!(endpoints.api, "https://api.github.com");
} }
fn atom_feed(tags: &[&str]) -> String {
let entries: String = tags
.iter()
.map(|t| {
format!(r#"<link rel="alternate" href="https://github.com/o/r/releases/tag/{t}"/>"#)
})
.collect();
format!("<feed>{entries}</feed>")
}
#[test]
fn latest_release_skips_tags_with_no_real_release() {
let mut server = mockito::Server::new();
let endpoints = GithubEndpoints {
web: server.url(),
api: server.url(),
};
// Mirrors the real scaleway-cli case: newest feed entry (a -dbg1
// tag) has no Release object behind it and 404s.
let _feed = server
.mock("GET", "/o/r/releases.atom")
.with_status(200)
.with_body(atom_feed(&["v2.62.0-dbg1", "v2.62.0"]))
.create();
let _missing = server
.mock("GET", "/repos/o/r/releases/tags/v2.62.0-dbg1")
.with_status(404)
.create();
let _real = server
.mock("GET", "/repos/o/r/releases/tags/v2.62.0")
.with_status(200)
.with_body("{}")
.create();
let client = reqwest::blocking::Client::new();
let tag = endpoints.latest_release(&client, "o/r").unwrap();
assert_eq!(tag, "v2.62.0");
}
#[test]
fn latest_release_errors_when_feed_has_no_tags() {
let mut server = mockito::Server::new();
let endpoints = GithubEndpoints {
web: server.url(),
api: server.url(),
};
let _feed = server
.mock("GET", "/o/r/releases.atom")
.with_status(200)
.with_body("<feed></feed>")
.create();
let client = reqwest::blocking::Client::new();
let err = endpoints.latest_release(&client, "o/r").unwrap_err();
assert!(err.to_string().contains("no release tag found"));
}
#[test]
fn latest_release_errors_when_no_candidate_resolves() {
let mut server = mockito::Server::new();
let endpoints = GithubEndpoints {
web: server.url(),
api: server.url(),
};
let _feed = server
.mock("GET", "/o/r/releases.atom")
.with_status(200)
.with_body(atom_feed(&["v1.0.0-dbg1"]))
.create();
let _missing = server
.mock("GET", "/repos/o/r/releases/tags/v1.0.0-dbg1")
.with_status(404)
.create();
let client = reqwest::blocking::Client::new();
let err = endpoints.latest_release(&client, "o/r").unwrap_err();
assert!(err.to_string().contains("resolved to a real release"));
}
#[test]
fn api_is_the_configured_api_root() {
let endpoints = GithubEndpoints {
web: "http://w".into(),
api: "http://a".into(),
};
assert_eq!(endpoints.api(), "http://a");
}
} }

1
src/main.rs
View file

@ -6,6 +6,7 @@ mod builder;
mod checker; mod checker;
mod config; mod config;
mod fetcher; mod fetcher;
mod forgejo;
mod github; mod github;
mod hash; mod hash;
mod notifier; mod notifier;

48
src/pipeline.rs
View file

@ -6,14 +6,14 @@
//! `main.rs`. //! `main.rs`.
use crate::builder; use crate::builder;
use crate::checker; use crate::checker::{self, ReleaseSource};
use crate::config::{self, Package}; use crate::config::{self, Package};
use crate::fetcher::{self, DownloadedAsset}; use crate::fetcher::{self, DownloadedAsset};
use crate::notifier::{self, Event}; use crate::notifier::{self, Event};
use crate::paths::Paths; use crate::paths::Paths;
use crate::publisher; use crate::publisher;
use crate::sanity; use crate::sanity;
use crate::source::Endpoints; use crate::source;
use crate::state; use crate::state;
use crate::verifier::{self, VerificationResult}; use crate::verifier::{self, VerificationResult};
use anyhow::{Context, Result, bail}; use anyhow::{Context, Result, bail};
@ -72,8 +72,15 @@ pub fn run_check() -> Result<()> {
let mut any_failed = false; let mut any_failed = false;
for (name, pkg) in &packages { for (name, pkg) in &packages {
println!("== {name} ({}) ==", pkg.repo); println!("== {name} ({}) ==", pkg.repo);
let result = Endpoints::for_package(pkg).and_then(|endpoints| { let result = source::for_package(pkg).and_then(|release_source| {
process_package(&client, &endpoints, &state_dir, &work_dir, name, pkg) process_package(
&client,
release_source.as_ref(),
&state_dir,
&work_dir,
name,
pkg,
)
}); });
if let Err(err) = result { if let Err(err) = result {
eprintln!(" error: {err:#}"); eprintln!(" error: {err:#}");
@ -120,13 +127,13 @@ fn decide_tier_action(tier: u8, passed: bool, already_pending_this_version: bool
fn process_package( fn process_package(
client: &reqwest::blocking::Client, client: &reqwest::blocking::Client,
endpoints: &Endpoints, release_source: &dyn ReleaseSource,
state_dir: &Path, state_dir: &Path,
work_dir: &Path, work_dir: &Path,
name: &str, name: &str,
pkg: &Package, pkg: &Package,
) -> Result<()> { ) -> Result<()> {
let latest = checker::latest_release(client, endpoints, &pkg.repo)?; let latest = release_source.latest_release(client, &pkg.repo)?;
let last_seen = state::load_last_version(state_dir, name); let last_seen = state::load_last_version(state_dir, name);
if last_seen.as_deref() == Some(latest.as_str()) { if last_seen.as_deref() == Some(latest.as_str()) {
println!(" up to date at {latest}"); println!(" up to date at {latest}");
@ -134,7 +141,7 @@ fn process_package(
} }
println!(" new version detected: {latest} (previously: {last_seen:?})"); println!(" new version detected: {latest} (previously: {last_seen:?})");
let fetched = fetch_and_verify(client, endpoints, work_dir, name, pkg, &latest)?; let fetched = fetch_and_verify(client, release_source, work_dir, name, pkg, &latest)?;
println!(" fetched {}", fetched.asset.path.display()); println!(" fetched {}", fetched.asset.path.display());
println!( println!(
" verification (tier {}): {} — {}", " verification (tier {}): {} — {}",
@ -207,7 +214,7 @@ struct FetchVerifyResult {
/// trusting a possibly-stale flag from an earlier run). /// trusting a possibly-stale flag from an earlier run).
fn fetch_and_verify( fn fetch_and_verify(
client: &reqwest::blocking::Client, client: &reqwest::blocking::Client,
endpoints: &Endpoints, release_source: &dyn ReleaseSource,
work_dir: &Path, work_dir: &Path,
name: &str, name: &str,
pkg: &Package, pkg: &Package,
@ -217,7 +224,7 @@ fn fetch_and_verify(
let asset_name = pkg.asset_pattern.replace("{version}", &version); let asset_name = pkg.asset_pattern.replace("{version}", &version);
let dest_dir = work_dir.join(name).join(tag); let dest_dir = work_dir.join(name).join(tag);
let api = endpoints.api(); let api = release_source.api();
let asset = fetcher::download_asset(client, api, &pkg.repo, tag, &asset_name, &dest_dir)?; let asset = fetcher::download_asset(client, api, &pkg.repo, tag, &asset_name, &dest_dir)?;
let verification = verifier::verify( let verification = verifier::verify(
client, client,
@ -320,11 +327,11 @@ pub fn run_review(args: &[String]) -> Result<()> {
fn approve(state_dir: &Path, work_dir: &Path, name: &str, pkg: &Package, tag: &str) -> Result<()> { fn approve(state_dir: &Path, work_dir: &Path, name: &str, pkg: &Package, tag: &str) -> Result<()> {
let client = build_client()?; let client = build_client()?;
let endpoints = Endpoints::for_package(pkg)?; let release_source = source::for_package(pkg)?;
// Re-verify rather than trusting the earlier flag: the artifact at // Re-verify rather than trusting the earlier flag: the artifact at
// this tag could in principle have changed since it was queued. // this tag could in principle have changed since it was queued.
let fetched = fetch_and_verify(&client, &endpoints, work_dir, name, pkg, tag)?; let fetched = fetch_and_verify(&client, release_source.as_ref(), work_dir, name, pkg, tag)?;
if !fetched.verification.passed { if !fetched.verification.passed {
bail!( bail!(
"re-verification failed on approve: {}", "re-verification failed on approve: {}",
@ -346,6 +353,7 @@ fn approve(state_dir: &Path, work_dir: &Path, name: &str, pkg: &Package, tag: &s
#[cfg(test)] #[cfg(test)]
mod tests { mod tests {
use super::*; use super::*;
use crate::forgejo::ForgejoEndpoints;
use crate::github::GithubEndpoints; use crate::github::GithubEndpoints;
use crate::test_support::same_origin_package; use crate::test_support::same_origin_package;
@ -381,7 +389,7 @@ mod tests {
/// Mocks the release-tag, asset, and checksum endpoints for `o/r@v1.0.0` /// Mocks the release-tag, asset, and checksum endpoints for `o/r@v1.0.0`
/// with a checksum that doesn't match the asset, so verification fails. /// with a checksum that doesn't match the asset, so verification fails.
/// These are identical on GitHub and Forgejo (see `Endpoints::api`); only /// These are identical on GitHub and Forgejo (see `ReleaseSource::api`); only
/// how the latest tag is found differs per test. Returned mocks must /// how the latest tag is found differs per test. Returned mocks must
/// stay alive for the test's duration. /// stay alive for the test's duration.
fn mock_release_with_bad_checksum(server: &mut mockito::ServerGuard) -> Vec<mockito::Mock> { fn mock_release_with_bad_checksum(server: &mut mockito::ServerGuard) -> Vec<mockito::Mock> {
@ -418,14 +426,14 @@ mod tests {
/// Runs `process_package` for a package whose checksum is wrong and /// Runs `process_package` for a package whose checksum is wrong and
/// asserts it errors with "verification failed" while leaving no trace /// asserts it errors with "verification failed" while leaving no trace
/// in state. /// in state.
fn assert_verification_failure_is_an_error(endpoints: &Endpoints, pkg: &Package) { fn assert_verification_failure_is_an_error(release_source: &dyn ReleaseSource, pkg: &Package) {
let client = reqwest::blocking::Client::new(); let client = reqwest::blocking::Client::new();
let state_dir = tempfile::tempdir().unwrap(); let state_dir = tempfile::tempdir().unwrap();
let work_dir = tempfile::tempdir().unwrap(); let work_dir = tempfile::tempdir().unwrap();
let err = process_package( let err = process_package(
&client, &client,
endpoints, release_source,
state_dir.path(), state_dir.path(),
work_dir.path(), work_dir.path(),
"thing", "thing",
@ -450,10 +458,10 @@ mod tests {
#[test] #[test]
fn process_package_returns_err_on_verification_failure() { fn process_package_returns_err_on_verification_failure() {
let mut server = mockito::Server::new(); let mut server = mockito::Server::new();
let endpoints = Endpoints::Github(GithubEndpoints { let github = GithubEndpoints {
web: server.url(), web: server.url(),
api: server.url(), api: server.url(),
}); };
let feed = format!( let feed = format!(
r#"<feed><link rel="alternate" href="{}/o/r/releases/tag/v1.0.0"/></feed>"#, r#"<feed><link rel="alternate" href="{}/o/r/releases/tag/v1.0.0"/></feed>"#,
server.url() server.url()
@ -465,7 +473,7 @@ mod tests {
.create(); .create();
let _release_mocks = mock_release_with_bad_checksum(&mut server); let _release_mocks = mock_release_with_bad_checksum(&mut server);
assert_verification_failure_is_an_error(&endpoints, &same_origin_package("")); assert_verification_failure_is_an_error(&github, &same_origin_package(""));
} }
/// Same as above but through a Forgejo source, proving the whole /// Same as above but through a Forgejo source, proving the whole
@ -474,7 +482,7 @@ mod tests {
#[test] #[test]
fn process_package_returns_err_on_verification_failure_via_forgejo() { fn process_package_returns_err_on_verification_failure_via_forgejo() {
let mut server = mockito::Server::new(); let mut server = mockito::Server::new();
let endpoints = Endpoints::Forgejo { api: server.url() }; let forgejo = ForgejoEndpoints { api: server.url() };
let _latest = server let _latest = server
.mock("GET", "/repos/o/r/releases/latest") .mock("GET", "/repos/o/r/releases/latest")
.with_status(200) .with_status(200)
@ -482,8 +490,8 @@ mod tests {
.create(); .create();
let _release_mocks = mock_release_with_bad_checksum(&mut server); let _release_mocks = mock_release_with_bad_checksum(&mut server);
// The package's own `source` is irrelevant here: `endpoints` is // The package's own `source` is irrelevant here: `forgejo` is
// hand-built to point at the mock server, bypassing `for_package`. // hand-built to point at the mock server, bypassing `for_package`.
assert_verification_failure_is_an_error(&endpoints, &same_origin_package("")); assert_verification_failure_is_an_error(&forgejo, &same_origin_package(""));
} }
} }

74
src/source.rs
View file

@ -1,46 +1,24 @@
//! Maps a package's configured `source` to the endpoints the pipeline //! Picks the `ReleaseSource` implementation for a package from its
//! stages talk to. `checker` and `fetcher` take what it hands them and //! configured `source`. The one place that knows which hosts exist; the
//! build their own paths under it. //! pipeline and the stages it calls only ever see the trait.
use crate::checker::ReleaseSource;
use crate::config::{Package, Source}; use crate::config::{Package, Source};
use crate::forgejo::ForgejoEndpoints;
use crate::github::GithubEndpoints; use crate::github::GithubEndpoints;
use anyhow::{Context, Result}; use anyhow::{Context, Result};
#[derive(Debug, Clone)] /// Defensive: errors only if a `forgejo-release` package has no
pub enum Endpoints { /// `base_url`, which `config::load_packages_dir` already guarantees.
Github(GithubEndpoints), pub fn for_package(pkg: &Package) -> Result<Box<dyn ReleaseSource>> {
/// A Forgejo/Gitea instance's API root, i.e. `<base_url>/api/v1`. match pkg.source {
Forgejo { Source::GithubRelease => Ok(Box::new(GithubEndpoints::default())),
api: String, Source::ForgejoRelease => {
}, let base_url = pkg
} .base_url
.as_deref()
impl Endpoints { .context("source = \"forgejo-release\" needs a base_url")?;
/// Defensive: errors only if a `forgejo-release` package has no Ok(Box::new(ForgejoEndpoints::from_base_url(base_url)))
/// `base_url`, which `config::load_packages_dir` already guarantees.
pub fn for_package(pkg: &Package) -> Result<Self> {
match pkg.source {
Source::GithubRelease => Ok(Endpoints::Github(GithubEndpoints::default())),
Source::ForgejoRelease => {
let base_url = pkg
.base_url
.as_deref()
.context("source = \"forgejo-release\" needs a base_url")?;
Ok(Endpoints::Forgejo {
api: format!("{}/api/v1", base_url.trim_end_matches('/')),
})
}
}
}
/// The releases API root. GitHub and Forgejo both serve
/// `/repos/{owner}/{repo}/releases/tags/{tag}` under it with the same
/// `assets[].{name, browser_download_url}` shape, which is why
/// `fetcher` and `verifier` need only this and not the enum.
pub fn api(&self) -> &str {
match self {
Endpoints::Github(github) => &github.api,
Endpoints::Forgejo { api } => api,
} }
} }
} }
@ -52,28 +30,20 @@ mod tests {
#[test] #[test]
fn github_source_uses_real_github() { fn github_source_uses_real_github() {
let endpoints = Endpoints::for_package(&package("")).unwrap(); let source = for_package(&package("")).unwrap();
assert_eq!(endpoints.api(), "https://api.github.com"); assert_eq!(source.api(), "https://api.github.com");
assert!(matches!(endpoints, Endpoints::Github(_)));
} }
#[test] #[test]
fn forgejo_source_appends_api_v1() { fn forgejo_source_uses_the_instances_api() {
let pkg = package("source = \"forgejo-release\"\nbase_url = \"https://code.example.com\""); let pkg = package("source = \"forgejo-release\"\nbase_url = \"https://code.example.com\"");
let endpoints = Endpoints::for_package(&pkg).unwrap(); let source = for_package(&pkg).unwrap();
assert_eq!(endpoints.api(), "https://code.example.com/api/v1"); assert_eq!(source.api(), "https://code.example.com/api/v1");
}
#[test]
fn forgejo_source_tolerates_trailing_slash() {
let pkg = package("source = \"forgejo-release\"\nbase_url = \"https://code.example.com/\"");
let endpoints = Endpoints::for_package(&pkg).unwrap();
assert_eq!(endpoints.api(), "https://code.example.com/api/v1");
} }
#[test] #[test]
fn forgejo_source_without_base_url_errors() { fn forgejo_source_without_base_url_errors() {
let err = Endpoints::for_package(&package("source = \"forgejo-release\"")).unwrap_err(); let err = for_package(&package("source = \"forgejo-release\"")).unwrap_err();
assert!(err.to_string().contains("needs a base_url")); assert!(err.to_string().contains("needs a base_url"));
} }
} }