# Spike: `ai-agents` (declarative YAML) vs. hand-wired `rig-core`

## What this proves

- `ai-agents` (crates.io `ai-agents` v1.0.0, Rust-native, no Python) can load an
  agent purely from YAML and talk to both of this project's real backends:
  - `spike/agents/writer.yaml` -> Ollama (`gemma4-e4b:latest`), matches `wire_gemma_client`
    in `src/main.rs`.
  - `spike/agents/critic.yaml` -> Ollama (`gemma4-e4b:latest`), matches the
    self-review/quality-guard call in `revise.rs`'s `is_usable()` — same
    provider as the writer, but its own system prompt and prompt template
    lifted verbatim from `prompts.toml`'s `[critic]` section.
  - `spike/agents/judge.yaml` -> `provider: openai-compatible` against the local
    `llama-server` (Shieldstral), matches `wire_shieldstral`.
  - All three worked verbatim against this machine's real models/config, no mocking.
- `spike/pipeline.yaml` expresses the three-stage generate -> critic -> judge
  flow (this project's `revise::generate_below_threshold` shape, including
  the format quality-guard before scoring) as one declarative `pipeline:`
  block with `spawner.auto_spawn` and `{{ stages.<id> }}` templating, no
  manual Rust orchestration code. Confirmed working end to end: writer runs,
  critic judges its format (`yes`/`no`), judge scores it independently — all
  three stages complete in a single `agent.chat()` call.

## What it doesn't prove (real limitations found)

1. ~~VRAM ceiling~~ **Resolved — not a framework limitation at all.** The
   initial failure (`cudaMalloc failed: out of memory` from Ollama's
   `/api/chat`) was from running Shieldstral's `llama-server` at its
   default `-ngl 999` (full GPU offload) alongside Ollama's own gemma
   load — both fighting for the same 8GB card. Restarting `llama-server`
   with `-ngl 0` (CPU-only, same flag this project already documents
   using for Shieldstral) puts it entirely on CPU, and the full
   three-stage `pipeline:` (writer -> critic -> judge) then runs cleanly
   end to end in one process — GPU usage stayed flat at ~6GB (all Ollama)
   throughout. This
   is a `llama-server` launch flag, not anything `ai-agents`-specific;
   `ai-agents` never touches GPU/CPU placement itself, it only talks HTTP
   to whatever backend is configured. `src/server.rs`'s `ensure_running()`
   doesn't currently pass `-ngl`, so it would need an `-ngl 0` addition
   (mirroring `server.toml`) to get the same behavior in the real app.
2. **No logprob-based scoring.** `revise.rs`'s real `score()` function reads
   token logprobs off Shieldstral's response (see `models::ChatLogprobs`) to
   get a continuous 0.0-1.0 score, not a yes/no string. `ai-agents`'
   `Agent::chat()` returns plain text content; there's no exposed hook for
   raw logprobs in the YAML/builder API surface I found. Reproducing the
   current scoring behavior would mean dropping to `ai-agents`' lower-level
   provider access (if any) or keeping rig-core for the judge call and only
   using `ai-agents` for orchestration/prompt config — a hybrid, not a
   clean swap.
3. **Critic stage runs but doesn't gate anything.** `spike/agents/critic.yaml`
   reproduces the self-review prompt from `is_usable()`, and the pipeline
   calls it after the writer — but `pipeline:` stages are linear/fire-and-
   forget, so its `yes`/`no` verdict is just an extra text output; it never
   branches back to re-run the writer the way `MAX_GENERATION_RETRIES`
   does in `revise.rs`. Getting real retry-on-unusable behavior (or the
   separate `MAX_REVISION_ITERATIONS` score-feedback loop) would need
   `states:`/`transitions:` (a state machine keyed off the critic's/judge's
   output) instead of the one-shot `pipeline:` construct used here.

## Verdict

The declarative-YAML story checks out for *provider wiring and prompt
config* — that part is genuinely config, not code, and matches the
CrewAI-style ergonomics from the earlier conversation, and the full
two-stage pipeline now runs end to end against this project's real local
models (see reproducing steps below). It does **not** cleanly cover this
project's actual judge mechanism (logprob scoring), so adopting it
wholesale would be a partial rewrite of `revise.rs`'s scoring logic, not a
drop-in replacement. Worth revisiting if a future judge model switches to
yes/no-only verdicts, or if `ai-agents` grows raw-logprob access.

## Reproducing

```
ollama serve                      # writer leg (gemma)

/home/austin/.local/share/llama.cpp/build/bin/llama-server \
  -m /home/austin/ai/Shieldstral-1.0-3B-BF16.gguf --jinja -c 32768 \
  --host 127.0.0.1 --port 8000 -ngl 0   # judge leg, CPU-only so it
                                         # doesn't fight gemma for VRAM

cargo run --bin ai_agents_spike
```