doubleo7/spike/NOTES.md

Spike: ai-agents (declarative YAML) vs. hand-wired rig-core

What this proves

• ai-agents (crates.io ai-agents v1.0.0, Rust-native, no Python) can load an agent purely from YAML and talk to both of this project's real backends:
  • spike/agents/writer.yaml -> Ollama (gemma4-e4b:latest), matches wire_gemma_client in src/main.rs.
  • spike/agents/critic.yaml -> Ollama (gemma4-e4b:latest), matches the self-review/quality-guard call in revise.rs's is_usable() — same provider as the writer, but its own system prompt and prompt template lifted verbatim from prompts.toml's [critic] section.
  • spike/agents/judge.yaml -> provider: openai-compatible against the local llama-server (Shieldstral), matches wire_shieldstral.
  • All three worked verbatim against this machine's real models/config, no mocking.
• spike/pipeline.yaml expresses the three-stage generate -> critic -> judge flow (this project's revise::generate_below_threshold shape, including the format quality-guard before scoring) as one declarative pipeline: block with spawner.auto_spawn and {{ stages.<id> }} templating, no manual Rust orchestration code. Confirmed working end to end: writer runs, critic judges its format (yes/no), judge scores it independently — all three stages complete in a single agent.chat() call.

What it doesn't prove (real limitations found)

1. VRAM ceiling Resolved — not a framework limitation at all. The initial failure (cudaMalloc failed: out of memory from Ollama's /api/chat) was from running Shieldstral's llama-server at its default -ngl 999 (full GPU offload) alongside Ollama's own gemma load — both fighting for the same 8GB card. Restarting llama-server with -ngl 0 (CPU-only, same flag this project already documents using for Shieldstral) puts it entirely on CPU, and the full three-stage pipeline: (writer -> critic -> judge) then runs cleanly end to end in one process — GPU usage stayed flat at ~6GB (all Ollama) throughout. This is a llama-server launch flag, not anything ai-agents-specific; ai-agents never touches GPU/CPU placement itself, it only talks HTTP to whatever backend is configured. src/server.rs's ensure_running() doesn't currently pass -ngl, so it would need an -ngl 0 addition (mirroring server.toml) to get the same behavior in the real app.
2. No logprob-based scoring. revise.rs's real score() function reads token logprobs off Shieldstral's response (see models::ChatLogprobs) to get a continuous 0.0-1.0 score, not a yes/no string. ai-agents' Agent::chat() returns plain text content; there's no exposed hook for raw logprobs in the YAML/builder API surface I found. Reproducing the current scoring behavior would mean dropping to ai-agents' lower-level provider access (if any) or keeping rig-core for the judge call and only using ai-agents for orchestration/prompt config — a hybrid, not a clean swap.
3. Critic stage runs but doesn't gate anything. spike/agents/critic.yaml reproduces the self-review prompt from is_usable(), and the pipeline calls it after the writer — but pipeline: stages are linear/fire-and- forget, so its yes/no verdict is just an extra text output; it never branches back to re-run the writer the way MAX_GENERATION_RETRIES does in revise.rs. Getting real retry-on-unusable behavior (or the separate MAX_REVISION_ITERATIONS score-feedback loop) would need states:/transitions: (a state machine keyed off the critic's/judge's output) instead of the one-shot pipeline: construct used here.

Verdict

The declarative-YAML story checks out for provider wiring and prompt config — that part is genuinely config, not code, and matches the CrewAI-style ergonomics from the earlier conversation, and the full two-stage pipeline now runs end to end against this project's real local models (see reproducing steps below). It does not cleanly cover this project's actual judge mechanism (logprob scoring), so adopting it wholesale would be a partial rewrite of revise.rs's scoring logic, not a drop-in replacement. Worth revisiting if a future judge model switches to yes/no-only verdicts, or if ai-agents grows raw-logprob access.

Reproducing

ollama serve                      # writer leg (gemma)

/home/austin/.local/share/llama.cpp/build/bin/llama-server \
  -m /home/austin/ai/Shieldstral-1.0-3B-BF16.gguf --jinja -c 32768 \
  --host 127.0.0.1 --port 8000 -ngl 0   # judge leg, CPU-only so it
                                         # doesn't fight gemma for VRAM

cargo run --bin ai_agents_spike