Tried reproducing MAX_GENERATION_RETRIES (critic rejects -> retry writer
-> re-check -> judge) using ai-agents' states:/transitions: instead of
pipeline:, since pipeline stages can't branch.
Two real blockers found and confirmed against the crate source:
- delegate: states have no per-turn input override (only pipeline:/
concurrent: stages get input: templates), so the critic just echoed
the document back instead of answering yes/no.
- extract: context extractors read the state's incoming user_message,
not its generated/delegated response (runtime.rs:7320), so a guard
meant to gate on "what the critic just said" has nothing real to
read -- the loop-back transition never fires.
Documented as a genuine finding in NOTES.md rather than forcing a
fragile demo: this crate's state machine is built for turn-based intent
routing, not gating on a sub-agent's structured verdict.
The pipeline was only writer->judge, missing the format quality-check
gemma does on its own output before scoring (is_usable() in revise.rs).
Added spike/agents/critic.yaml with the [critic] prompts from
prompts.toml, wired as a middle pipeline stage. Confirmed the full
three-stage pipeline runs end to end. Note in NOTES.md that the stage
runs but doesn't gate/retry -- pipeline: is linear, real retry-on-unusable
behavior would need states:/transitions:.
The earlier VRAM-exhaustion failure was llama-server defaulting to full
GPU offload for Shieldstral while Ollama's gemma was also resident.
Restarting llama-server with -ngl 0 frees the GPU for gemma and the
full writer->judge ai-agents pipeline runs end to end with no VRAM
contention. Not an ai-agents limitation -- it never touches GPU
placement, only HTTP.
Adds a standalone binary (src/bin/ai_agents_spike.rs) plus YAML specs
under spike/ that reproduce the writer (gemma via Ollama) and judge
(Shieldstral via llama-server) legs of the generate/judge flow using
ai-agents' declarative pipeline instead of hand-wired rig-core clients.
Both legs verified working individually against real local models.
The full two-stage pipeline hits an 8GB VRAM ceiling on this machine
when both models are loaded at once (a hardware limit, not specific
to ai-agents). ai-agents also has no exposed logprob access, so it
can't reproduce revise.rs's actual scoring mechanism as-is. See
spike/NOTES.md for the full writeup and verdict.
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
