Add a Forgejo release source #5
10 changed files with 25 additions and 31 deletions
|
|
@ -35,8 +35,8 @@ practice rather than asserted from habit — see Further reading.
|
||||||
`helpers/`) tends toward the opposite, and a single feature change ends
|
`helpers/`) tends toward the opposite, and a single feature change ends
|
||||||
up touching files scattered across every layer.
|
up touching files scattered across every layer.
|
||||||
**Rule**: modules are named after what they do in the pipeline
|
**Rule**: modules are named after what they do in the pipeline
|
||||||
(`source`, `fetcher`, `verifier`, `builder`, `sanity`, `publisher`,
|
(`release_source`, `fetcher`, `verifier`, `builder`, `sanity`,
|
||||||
`state`), not generic buckets. (`source` is a directory module: the
|
`publisher`, `state`), not generic buckets. (`release_source` is a directory module: the
|
||||||
`ReleaseSource` trait and one file per host, re-exported from its
|
`ReleaseSource` trait and one file per host, re-exported from its
|
||||||
`mod.rs` so the rest of the crate never names a host's file.) A new pipeline stage gets a new module
|
`mod.rs` so the rest of the crate never names a host's file.) A new pipeline stage gets a new module
|
||||||
named after the stage, not a method bolted onto an existing one.
|
named after the stage, not a method bolted onto an existing one.
|
||||||
|
|
@ -109,7 +109,7 @@ practice rather than asserted from habit — see Further reading.
|
||||||
directory, but the same information — why this way and not the obvious
|
directory, but the same information — why this way and not the obvious
|
||||||
alternative — needs to live somewhere a future reader will actually see
|
alternative — needs to live somewhere a future reader will actually see
|
||||||
it: the doc comment on the thing itself.
|
it: the doc comment on the thing itself.
|
||||||
**Example already here**: `source/github.rs`'s doc comment on
|
**Example already here**: `release_source/github.rs`'s doc comment on
|
||||||
`GithubEndpoints::latest_release` explains why the newest Atom-feed entry isn't
|
`GithubEndpoints::latest_release` explains why the newest Atom-feed entry isn't
|
||||||
trusted outright (scaleway-cli's `-dbg1` tag has no real Release behind
|
trusted outright (scaleway-cli's `-dbg1` tag has no real Release behind
|
||||||
it) — the reasoning lives right next to the code it justifies, not in a
|
it) — the reasoning lives right next to the code it justifies, not in a
|
||||||
|
|
|
||||||
|
|
@ -373,12 +373,12 @@ Open questions on the schema:
|
||||||
likely reuses `nvchecker`'s logic/sources conceptually for non-GitHub
|
likely reuses `nvchecker`'s logic/sources conceptually for non-GitHub
|
||||||
sources eventually. For GitHub sources, prefers the `github-atom` feed
|
sources eventually. For GitHub sources, prefers the `github-atom` feed
|
||||||
(see Scaling > Check method) over unconditional REST polling.
|
(see Scaling > Check method) over unconditional REST polling.
|
||||||
*(Implemented for GitHub and Forgejo. `src/source/` defines the
|
*(Implemented for GitHub and Forgejo. `src/release_source/` defines the
|
||||||
`ReleaseSource` trait (latest release + API root); each host implements
|
`ReleaseSource` trait (latest release + API root); each host implements
|
||||||
it in its own file (`github.rs`, `forgejo.rs`), and the module's
|
it in its own file (`github.rs`, `forgejo.rs`), and the module's
|
||||||
`for_package` picks one per package, so adding a host doesn't touch
|
`for_package` picks one per package, so adding a host doesn't touch
|
||||||
existing ones. The rest of the crate imports the trait and hosts from
|
existing ones. The rest of the crate imports the trait and hosts from
|
||||||
`crate::source`, which re-exports them. A trait
|
`crate::release_source`, which re-exports them. A trait
|
||||||
rather than an enum match because there are now two real hosts with
|
rather than an enum match because there are now two real hosts with
|
||||||
genuinely different logic. GitHub regex-matches the first
|
genuinely different logic. GitHub regex-matches the first
|
||||||
`releases/tag/<tag>` link in the feed rather than doing a full XML
|
`releases/tag/<tag>` link in the feed rather than doing a full XML
|
||||||
|
|
|
||||||
|
|
@ -1,5 +1,5 @@
|
||||||
//! Turns a release tag into a version string. What the latest tag *is*
|
//! Turns a release tag into a version string. What the latest tag *is*
|
||||||
//! comes from a `ReleaseSource` (see `source`); this is the one piece of
|
//! comes from a `ReleaseSource` (see `release_source`); this is the one piece of
|
||||||
//! the check stage that isn't host-specific.
|
//! the check stage that isn't host-specific.
|
||||||
|
|
||||||
/// Strips a leading `v` from a release tag, e.g. `v2.62.0` -> `2.62.0`.
|
/// Strips a leading `v` from a release tag, e.g. `v2.62.0` -> `2.62.0`.
|
||||||
|
|
|
||||||
|
|
@ -1,6 +1,6 @@
|
||||||
//! Downloads a named release asset (from GitHub or Forgejo) to a local
|
//! Downloads a named release asset (from GitHub or Forgejo) to a local
|
||||||
//! path. The only module that talks to the releases API for asset bytes —
|
//! path. The only module that talks to the releases API for asset bytes —
|
||||||
//! `source` only resolves version tags, never downloads.
|
//! `release_source` only resolves version tags, never downloads.
|
||||||
|
|
||||||
use anyhow::{Context, Result};
|
use anyhow::{Context, Result};
|
||||||
use serde::Deserialize;
|
use serde::Deserialize;
|
||||||
|
|
|
||||||
|
|
@ -11,8 +11,8 @@ mod notifier;
|
||||||
mod paths;
|
mod paths;
|
||||||
mod pipeline;
|
mod pipeline;
|
||||||
mod publisher;
|
mod publisher;
|
||||||
|
mod release_source;
|
||||||
mod sanity;
|
mod sanity;
|
||||||
mod source;
|
|
||||||
mod state;
|
mod state;
|
||||||
#[cfg(test)]
|
#[cfg(test)]
|
||||||
mod test_support;
|
mod test_support;
|
||||||
|
|
|
||||||
|
|
@ -12,8 +12,8 @@ use crate::fetcher::{self, DownloadedAsset};
|
||||||
use crate::notifier::{self, Event};
|
use crate::notifier::{self, Event};
|
||||||
use crate::paths::Paths;
|
use crate::paths::Paths;
|
||||||
use crate::publisher;
|
use crate::publisher;
|
||||||
|
use crate::release_source::{self, ReleaseSource};
|
||||||
use crate::sanity;
|
use crate::sanity;
|
||||||
use crate::source::{self, ReleaseSource};
|
|
||||||
use crate::state;
|
use crate::state;
|
||||||
use crate::verifier::{self, VerificationResult};
|
use crate::verifier::{self, VerificationResult};
|
||||||
use anyhow::{Context, Result, bail};
|
use anyhow::{Context, Result, bail};
|
||||||
|
|
@ -72,15 +72,8 @@ pub fn run_check() -> Result<()> {
|
||||||
let mut any_failed = false;
|
let mut any_failed = false;
|
||||||
for (name, pkg) in &packages {
|
for (name, pkg) in &packages {
|
||||||
println!("== {name} ({}) ==", pkg.repo);
|
println!("== {name} ({}) ==", pkg.repo);
|
||||||
let result = source::for_package(pkg).and_then(|release_source| {
|
let result = release_source::for_package(pkg).and_then(|host| {
|
||||||
process_package(
|
process_package(&client, host.as_ref(), &state_dir, &work_dir, name, pkg)
|
||||||
&client,
|
|
||||||
release_source.as_ref(),
|
|
||||||
&state_dir,
|
|
||||||
&work_dir,
|
|
||||||
name,
|
|
||||||
pkg,
|
|
||||||
)
|
|
||||||
});
|
});
|
||||||
if let Err(err) = result {
|
if let Err(err) = result {
|
||||||
eprintln!(" error: {err:#}");
|
eprintln!(" error: {err:#}");
|
||||||
|
|
@ -127,13 +120,13 @@ fn decide_tier_action(tier: u8, passed: bool, already_pending_this_version: bool
|
||||||
|
|
||||||
fn process_package(
|
fn process_package(
|
||||||
client: &reqwest::blocking::Client,
|
client: &reqwest::blocking::Client,
|
||||||
release_source: &dyn ReleaseSource,
|
host: &dyn ReleaseSource,
|
||||||
state_dir: &Path,
|
state_dir: &Path,
|
||||||
work_dir: &Path,
|
work_dir: &Path,
|
||||||
name: &str,
|
name: &str,
|
||||||
pkg: &Package,
|
pkg: &Package,
|
||||||
) -> Result<()> {
|
) -> Result<()> {
|
||||||
let latest = release_source.latest_release(client, &pkg.repo)?;
|
let latest = host.latest_release(client, &pkg.repo)?;
|
||||||
let last_seen = state::load_last_version(state_dir, name);
|
let last_seen = state::load_last_version(state_dir, name);
|
||||||
if last_seen.as_deref() == Some(latest.as_str()) {
|
if last_seen.as_deref() == Some(latest.as_str()) {
|
||||||
println!(" up to date at {latest}");
|
println!(" up to date at {latest}");
|
||||||
|
|
@ -141,7 +134,7 @@ fn process_package(
|
||||||
}
|
}
|
||||||
println!(" new version detected: {latest} (previously: {last_seen:?})");
|
println!(" new version detected: {latest} (previously: {last_seen:?})");
|
||||||
|
|
||||||
let fetched = fetch_and_verify(client, release_source, work_dir, name, pkg, &latest)?;
|
let fetched = fetch_and_verify(client, host, work_dir, name, pkg, &latest)?;
|
||||||
println!(" fetched {}", fetched.asset.path.display());
|
println!(" fetched {}", fetched.asset.path.display());
|
||||||
println!(
|
println!(
|
||||||
" verification (tier {}): {} — {}",
|
" verification (tier {}): {} — {}",
|
||||||
|
|
@ -214,7 +207,7 @@ struct FetchVerifyResult {
|
||||||
/// trusting a possibly-stale flag from an earlier run).
|
/// trusting a possibly-stale flag from an earlier run).
|
||||||
fn fetch_and_verify(
|
fn fetch_and_verify(
|
||||||
client: &reqwest::blocking::Client,
|
client: &reqwest::blocking::Client,
|
||||||
release_source: &dyn ReleaseSource,
|
host: &dyn ReleaseSource,
|
||||||
work_dir: &Path,
|
work_dir: &Path,
|
||||||
name: &str,
|
name: &str,
|
||||||
pkg: &Package,
|
pkg: &Package,
|
||||||
|
|
@ -224,7 +217,7 @@ fn fetch_and_verify(
|
||||||
let asset_name = pkg.asset_pattern.replace("{version}", &version);
|
let asset_name = pkg.asset_pattern.replace("{version}", &version);
|
||||||
let dest_dir = work_dir.join(name).join(tag);
|
let dest_dir = work_dir.join(name).join(tag);
|
||||||
|
|
||||||
let api = release_source.api();
|
let api = host.api();
|
||||||
let asset = fetcher::download_asset(client, api, &pkg.repo, tag, &asset_name, &dest_dir)?;
|
let asset = fetcher::download_asset(client, api, &pkg.repo, tag, &asset_name, &dest_dir)?;
|
||||||
let verification = verifier::verify(
|
let verification = verifier::verify(
|
||||||
client,
|
client,
|
||||||
|
|
@ -327,11 +320,11 @@ pub fn run_review(args: &[String]) -> Result<()> {
|
||||||
|
|
||||||
fn approve(state_dir: &Path, work_dir: &Path, name: &str, pkg: &Package, tag: &str) -> Result<()> {
|
fn approve(state_dir: &Path, work_dir: &Path, name: &str, pkg: &Package, tag: &str) -> Result<()> {
|
||||||
let client = build_client()?;
|
let client = build_client()?;
|
||||||
let release_source = source::for_package(pkg)?;
|
let host = release_source::for_package(pkg)?;
|
||||||
|
|
||||||
// Re-verify rather than trusting the earlier flag: the artifact at
|
// Re-verify rather than trusting the earlier flag: the artifact at
|
||||||
// this tag could in principle have changed since it was queued.
|
// this tag could in principle have changed since it was queued.
|
||||||
let fetched = fetch_and_verify(&client, release_source.as_ref(), work_dir, name, pkg, tag)?;
|
let fetched = fetch_and_verify(&client, host.as_ref(), work_dir, name, pkg, tag)?;
|
||||||
if !fetched.verification.passed {
|
if !fetched.verification.passed {
|
||||||
bail!(
|
bail!(
|
||||||
"re-verification failed on approve: {}",
|
"re-verification failed on approve: {}",
|
||||||
|
|
@ -353,7 +346,7 @@ fn approve(state_dir: &Path, work_dir: &Path, name: &str, pkg: &Package, tag: &s
|
||||||
#[cfg(test)]
|
#[cfg(test)]
|
||||||
mod tests {
|
mod tests {
|
||||||
use super::*;
|
use super::*;
|
||||||
use crate::source::{ForgejoEndpoints, GithubEndpoints};
|
use crate::release_source::{ForgejoEndpoints, GithubEndpoints};
|
||||||
use crate::test_support::same_origin_package;
|
use crate::test_support::same_origin_package;
|
||||||
|
|
||||||
#[test]
|
#[test]
|
||||||
|
|
@ -425,14 +418,14 @@ mod tests {
|
||||||
/// Runs `process_package` for a package whose checksum is wrong and
|
/// Runs `process_package` for a package whose checksum is wrong and
|
||||||
/// asserts it errors with "verification failed" while leaving no trace
|
/// asserts it errors with "verification failed" while leaving no trace
|
||||||
/// in state.
|
/// in state.
|
||||||
fn assert_verification_failure_is_an_error(release_source: &dyn ReleaseSource, pkg: &Package) {
|
fn assert_verification_failure_is_an_error(host: &dyn ReleaseSource, pkg: &Package) {
|
||||||
let client = reqwest::blocking::Client::new();
|
let client = reqwest::blocking::Client::new();
|
||||||
let state_dir = tempfile::tempdir().unwrap();
|
let state_dir = tempfile::tempdir().unwrap();
|
||||||
let work_dir = tempfile::tempdir().unwrap();
|
let work_dir = tempfile::tempdir().unwrap();
|
||||||
|
|
||||||
let err = process_package(
|
let err = process_package(
|
||||||
&client,
|
&client,
|
||||||
release_source,
|
host,
|
||||||
state_dir.path(),
|
state_dir.path(),
|
||||||
work_dir.path(),
|
work_dir.path(),
|
||||||
"thing",
|
"thing",
|
||||||
|
|
|
||||||
|
|
@ -2,15 +2,16 @@
|
||||||
//! one file per host implementing it, and `for_package`, which picks the
|
//! one file per host implementing it, and `for_package`, which picks the
|
||||||
//! implementation for a package from its configured `source`. The
|
//! implementation for a package from its configured `source`. The
|
||||||
//! submodules are private and re-exported here, so the rest of the crate
|
//! submodules are private and re-exported here, so the rest of the crate
|
||||||
//! imports everything from `crate::source` and never a host's file.
|
//! imports everything from `crate::release_source` and never a host's
|
||||||
|
//! file.
|
||||||
|
|
||||||
|
mod contract;
|
||||||
mod forgejo;
|
mod forgejo;
|
||||||
mod github;
|
mod github;
|
||||||
mod release_source;
|
|
||||||
|
|
||||||
|
pub use contract::ReleaseSource;
|
||||||
pub use forgejo::ForgejoEndpoints;
|
pub use forgejo::ForgejoEndpoints;
|
||||||
pub use github::GithubEndpoints;
|
pub use github::GithubEndpoints;
|
||||||
pub use release_source::ReleaseSource;
|
|
||||||
|
|
||||||
use crate::config::{Package, Source};
|
use crate::config::{Package, Source};
|
||||||
use anyhow::{Context, Result};
|
use anyhow::{Context, Result};
|
||||||
Loading…
Reference in a new issue