schaefera/sporah-cli
Watch
1
0
Fork
You've already forked sporah-cli
 0
Code Issues 1 Pull requests 1 Projects Releases Packages Wiki Activity Actions

E2E tests against a mockito stub API, testing tiers #14

Merged
schaefera merged 6 commits from feat/testing-tiers into master 2026-10-17 09:33:39 +00:00
Conversation 3 Commits 6 Files changed 5 +501 -7
claude-bot commented 2026-10-16 20:31:13 +00:00
Collaborator

What

tests/e2e.rs runs the real sporah binary against a mockito stub of the API. It checks the exact output, request paths and exit codes for:

  • city and coords
  • --url vs SPORAH_URL
  • --values-only
  • 204, 400 and 502 responses
  • bad JSON

An ignored test checks the live API. There's a new CLAUDE.md with commands, Blast radius and Testing tiers, and a PR template.

On review request, the hand-written TcpListener stub is replaced by mockito 1.7.2, a dev-dependency only. I chose mockito because it has a sync Server::new(), so the tests stay plain #[test]. Each test declares the exact path and query it expects; any other request gets a 501.

Blast radius

Tier: trunk (Cargo.toml / Cargo.lock, for the mockito dev-dependency)
Trunk paths touched: Cargo.toml, Cargo.lock
Testing tier: standard (also: ship on sight, docs)
Gate / rollback: tests and docs only. The release dependency graph is unchanged (cargo tree -e normal,build --target all is byte-identical). To roll back, revert.

Proof

  • Suite: cargo test passes 9 unit and 8 e2e tests. The live api.sporah.eu test passed (run with cargo test -- --ignored).
  • Fails without the behaviour:
    • Setting MAX_BARS to 4 fails 2 tests.
    • Dropping the BAD_REQUEST arm fails no_content_and_bad_request_mean_no_data.
    • Dropping the trailing-slash trim fails url_flag_beats_env_and_drops_a_trailing_slash.
    • After the mockito switch, swapping lat and lon in the coords mock fails coords_query_the_root_with_lat_and_lon.
  • CI on f2d9dff: CI / ci (2m0s) and CI / audit (12s) are green (run 14).

Review

Reviewer:

  • The original suite got a fresh-context review at medium: 5 minor findings, 4 fixed, 1 declined.
  • The mockito switch got a fresh-context review at high. It checked mockito's matching in its source and diffed the release graph. It found 1 minor issue (stale "one-shot stub" wording in CLAUDE.md), fixed in f2d9dff.

Review-Confidence: 100/100 for the mockito delta, with no deductions. The earlier part was leaf and not audited.

Confidence by part

  • tests/e2e.rs: high. It uses exact-string asserts and was mutation-checked.
  • Cargo.toml / Cargo.lock: high. mockito is dev-only and the release graph is verified identical.
  • CLAUDE.md Testing tiers: medium. The waybar mode and release gates are written down but not yet exercised.

🤖 Generated with Claude Code

## What `tests/e2e.rs` runs the real `sporah` binary against a [mockito](https://crates.io/crates/mockito) stub of the API. It checks the exact output, request paths and exit codes for: - `city` and `coords` - `--url` vs `SPORAH_URL` - `--values-only` - 204, 400 and 502 responses - bad JSON An ignored test checks the live API. There's a new `CLAUDE.md` with commands, Blast radius and Testing tiers, and a PR template. On review request, the hand-written TcpListener stub is replaced by mockito 1.7.2, a dev-dependency only. I chose mockito because it has a sync `Server::new()`, so the tests stay plain `#[test]`. Each test declares the exact path and query it expects; any other request gets a 501. ## Blast radius **Tier:** trunk (Cargo.toml / Cargo.lock, for the mockito dev-dependency) **Trunk paths touched:** Cargo.toml, Cargo.lock **Testing tier:** standard (also: ship on sight, docs) **Gate / rollback:** tests and docs only. The release dependency graph is unchanged (`cargo tree -e normal,build --target all` is byte-identical). To roll back, revert. ## Proof - Suite: `cargo test` passes 9 unit and 8 e2e tests. The live `api.sporah.eu` test passed (run with `cargo test -- --ignored`). - Fails without the behaviour: - Setting MAX_BARS to 4 fails 2 tests. - Dropping the BAD_REQUEST arm fails `no_content_and_bad_request_mean_no_data`. - Dropping the trailing-slash trim fails `url_flag_beats_env_and_drops_a_trailing_slash`. - After the mockito switch, swapping lat and lon in the coords mock fails `coords_query_the_root_with_lat_and_lon`. - CI on f2d9dff: CI / ci (2m0s) and CI / audit (12s) are green (run 14). ## Review **Reviewer:** - The original suite got a fresh-context review at medium: 5 minor findings, 4 fixed, 1 declined. - The mockito switch got a fresh-context review at high. It checked mockito's matching in its source and diffed the release graph. It found 1 minor issue (stale "one-shot stub" wording in CLAUDE.md), fixed in f2d9dff. **Review-Confidence:** 100/100 for the mockito delta, with no deductions. The earlier part was leaf and not audited. ## Confidence by part - `tests/e2e.rs`: high. It uses exact-string asserts and was mutation-checked. - Cargo.toml / Cargo.lock: high. mockito is dev-only and the release graph is verified identical. - `CLAUDE.md` Testing tiers: medium. The waybar mode and release gates are written down but not yet exercised. 🤖 Generated with [Claude Code](https://claude.com/claude-code)
test: run the binary end to end against a stub API, add testing tiers
All checks were successful
CI / audit (pull_request) Successful in 11s
Details
CI / ci (pull_request) Successful in 34s
Details
717724813f 
tests/e2e.rs starts a one-shot HTTP stub per test and checks the real
binary's output and exit code for bars, coordinates, percent-encoded
city names, --values-only, 204/400 no-data, 5xx and malformed JSON. An
ignored test checks the contract against the live API.

CLAUDE.md gets commands, Blast radius and Testing tiers sections, and
the repo gets the shipping-a-pr PR template.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
fix: address review of the E2E tests and tiers
All checks were successful
CI / audit (pull_request) Successful in 10s
Details
CI / ci (pull_request) Successful in 31s
Details
749acc7e37 
Test that --url beats SPORAH_URL and loses a trailing slash. CLAUDE.md:
help text is trunk (src/cli.rs), the suite covers one-shot runs only, and
the live check can fail for reasons outside the change.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
chore: record review result
All checks were successful
CI / ci (pull_request) Successful in 38s
Details
CI / audit (pull_request) Successful in 12s
Details
d604cf7d31 
Review-Tier: leaf
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
claude-bot changed title from WIP: E2E tests against a stub API, testing tiers to E2E tests against a stub API, testing tiers 2026-10-16 20:40:37 +00:00
schaefera reviewed 2026-10-17 09:02:44 +00:00
tests/e2e.rs Outdated
@ -0,0 +14,4 @@
/// Serves one request with `status` and `body`, and returns the stub's base
/// URL plus a receiver for the request target the client sent.
fn stub(status: &'static str, body: &'static str) -> (String, mpsc::Receiver<String>) {
schaefera commented 2026-10-17 09:02:44 +00:00
Owner

Is there any crate or library which can more quickly stub or serve as a mock server than hand writing server code?

Is there any crate or library which can more quickly stub or serve as a mock server than hand writing server code?
test: stub the API with mockito instead of a hand-written server
Some checks failed
CI / audit (pull_request) Successful in 14s
Details
CI / ci (pull_request) Has been cancelled
Details
f77b0d717d 
Each test now declares the exact path it expects; a wrong path gets
mockito's 501 and the test fails. mockito is a dev-dependency only, so
the release binary and its static musl build are unchanged.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
docs: the E2E stub is mockito now
All checks were successful
CI / audit (pull_request) Successful in 12s
Details
CI / ci (pull_request) Successful in 2m0s
Details
f2d9dff3b1 
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
claude-bot reviewed 2026-10-17 09:07:21 +00:00
claude-bot left a comment

Replied inline on tests/e2e.rs.

Replied inline on tests/e2e.rs.
tests/e2e.rs
@ -0,0 +11,4 @@
/// A stub that answers `GET {path}` with `status` and `body`. Any other
/// request gets mockito's 501, and `Mock::assert` fails if `path` was never hit.
fn stub(path: &str, status: usize, body: &str) -> (ServerGuard, Mock) {
claude-bot commented 2026-10-17 09:07:21 +00:00
Author
Collaborator

Yes. In f77b0d7 the hand-written stub is replaced with mockito 1.7.2, a dev-dependency only. I picked it over wiremock and httpmock because it has a sync Server::new(), so the tests stay plain #[test] with no tokio runtime.

Each test now declares the exact path it expects (/Berlin, /Rothenburg%20ob%20der%20Tauber, /?lat=52.52&lon=13.4). mockito compares that against the full path and query, and any other request gets a 501, so the old request-target checks still hold. The channel plumbing is gone, and the file is about 40 lines shorter.

Checks:

  • Swapping lat and lon in the coords mock makes that test fail.
  • cargo tree -e normal,build is byte-identical before and after, so the release and musl build don't change.
Yes. In f77b0d7 the hand-written stub is replaced with [mockito](https://crates.io/crates/mockito) 1.7.2, a dev-dependency only. I picked it over wiremock and httpmock because it has a sync `Server::new()`, so the tests stay plain `#[test]` with no tokio runtime. Each test now declares the exact path it expects (`/Berlin`, `/Rothenburg%20ob%20der%20Tauber`, `/?lat=52.52&lon=13.4`). mockito compares that against the full path and query, and any other request gets a 501, so the old request-target checks still hold. The channel plumbing is gone, and the file is about 40 lines shorter. Checks: - Swapping lat and lon in the coords mock makes that test fail. - `cargo tree -e normal,build` is byte-identical before and after, so the release and musl build don't change.
schaefera marked this conversation as resolved
chore: record review result
All checks were successful
CI / audit (pull_request) Successful in 14s
Details
CI / ci (pull_request) Successful in 1m3s
Details
065a95c5fc 
Review-Confidence: 100
Review-Tier: trunk
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
claude-bot changed title from E2E tests against a stub API, testing tiers to E2E tests against a mockito stub API, testing tiers 2026-10-17 09:09:20 +00:00
schaefera approved these changes 2026-10-17 09:33:35 +00:00
schaefera merged commit 632c7a8547 into master 2026-10-17 09:33:39 +00:00
schaefera deleted branch feat/testing-tiers 2026-10-17 09:33:39 +00:00
Sign in to join this conversation.
Reviewers
No reviewers
schaefera
Labels
Clear labels
No items
No labels
Milestone
Clear milestone
No items
No milestone
Projects
Clear projects
No items
No project
Assignees
Clear assignees
No assignees
2 participants
Notifications
Due date
The due date is invalid or out of range. Please use the format "yyyy-mm-dd".

No due date set.

Dependencies

No dependencies set

Reference
schaefera/sporah-cli!14
No description provided.