schaefera/doubleo7
1
0
Fork 0
Code Issues Pull requests 1 Projects Releases Packages Wiki Activity Actions

WIP: Spike: ai-agents crate for declarative YAML agent config #1

Closed
claude-bot wants to merge 4 commits from spike/ai-agents-yaml-config into master
pull from: spike/ai-agents-yaml-config
merge into: schaefera:master
Conversation 1 Commits 4 Files changed 10 +1403 -16
claude-bot commented 2026-08-06 07:40:29 +00:00
Collaborator
Copy link

What

Tries the ai-agents crate (Rust-native, declarative YAML agent config, v1.0.0 as of 2026-08-02) as a potential replacement for the hand-wired rig-core clients in main.rs/revise.rs.

  • spike/agents/writer.yaml — the gemma/Ollama writer leg
  • spike/agents/judge.yaml — the Shieldstral/llama-server judge leg (provider: openai-compatible)
  • spike/pipeline.yaml — both wired into one declarative pipeline: (writer -> judge)
  • src/bin/ai_agents_spike.rs — runs it

Findings

Both legs verified working individually against this machine's real local models — genuinely config-driven provider wiring, no mocking. Full writeup and verdict in spike/NOTES.md.

Two real limitations found, not resolved here:

  1. The full two-stage pipeline needs both models resident in VRAM at once, which overflows this machine's 8GB card alongside Shieldstral's 32k-context llama-server — a hardware ceiling, same for the existing rig-core code, not specific to ai-agents.
  2. ai-agents' Agent::chat() returns plain text; there's no exposed logprob access, so it can't reproduce revise.rs's actual score() mechanism (which reads token logprobs off Shieldstral's response). Adopting this wholesale would mean a partial rewrite of the scoring logic, not a drop-in swap.

Draft — this is exploratory, not meant to merge as-is.

🤖 Generated with Claude Code

## What Tries the `ai-agents` crate (Rust-native, declarative YAML agent config, v1.0.0 as of 2026-08-02) as a potential replacement for the hand-wired `rig-core` clients in `main.rs`/`revise.rs`. - `spike/agents/writer.yaml` — the gemma/Ollama writer leg - `spike/agents/judge.yaml` — the Shieldstral/llama-server judge leg (`provider: openai-compatible`) - `spike/pipeline.yaml` — both wired into one declarative `pipeline:` (writer -> judge) - `src/bin/ai_agents_spike.rs` — runs it ## Findings Both legs verified working individually against this machine's real local models — genuinely config-driven provider wiring, no mocking. Full writeup and verdict in `spike/NOTES.md`. Two real limitations found, not resolved here: 1. The full two-stage pipeline needs both models resident in VRAM at once, which overflows this machine's 8GB card alongside Shieldstral's 32k-context llama-server — a hardware ceiling, same for the existing rig-core code, not specific to `ai-agents`. 2. `ai-agents`' `Agent::chat()` returns plain text; there's no exposed logprob access, so it can't reproduce `revise.rs`'s actual score() mechanism (which reads token logprobs off Shieldstral's response). Adopting this wholesale would mean a partial rewrite of the scoring logic, not a drop-in swap. Draft — this is exploratory, not meant to merge as-is. 🤖 Generated with [Claude Code](https://claude.com/claude-code)
claude-bot added 1 commit 2026-08-06 07:40:29 +00:00 
Adds a standalone binary (src/bin/ai_agents_spike.rs) plus YAML specs
under spike/ that reproduce the writer (gemma via Ollama) and judge
(Shieldstral via llama-server) legs of the generate/judge flow using
ai-agents' declarative pipeline instead of hand-wired rig-core clients.

Both legs verified working individually against real local models.
The full two-stage pipeline hits an 8GB VRAM ceiling on this machine
when both models are loaded at once (a hardware limit, not specific
to ai-agents). ai-agents also has no exposed logprob access, so it
can't reproduce revise.rs's actual scoring mechanism as-is. See
spike/NOTES.md for the full writeup and verdict.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
schaefera added 1 commit 2026-08-06 08:15:20 +00:00 
The earlier VRAM-exhaustion failure was llama-server defaulting to full
GPU offload for Shieldstral while Ollama's gemma was also resident.
Restarting llama-server with -ngl 0 frees the GPU for gemma and the
full writer->judge ai-agents pipeline runs end to end with no VRAM
contention. Not an ai-agents limitation -- it never touches GPU
placement, only HTTP.
schaefera added 1 commit 2026-08-06 08:17:57 +00:00 
The pipeline was only writer->judge, missing the format quality-check
gemma does on its own output before scoring (is_usable() in revise.rs).
Added spike/agents/critic.yaml with the [critic] prompts from
prompts.toml, wired as a middle pipeline stage. Confirmed the full
three-stage pipeline runs end to end. Note in NOTES.md that the stage
runs but doesn't gate/retry -- pipeline: is linear, real retry-on-unusable
behavior would need states:/transitions:.
schaefera added 1 commit 2026-08-06 08:26:24 +00:00 
Tried reproducing MAX_GENERATION_RETRIES (critic rejects -> retry writer
-> re-check -> judge) using ai-agents' states:/transitions: instead of
pipeline:, since pipeline stages can't branch.

Two real blockers found and confirmed against the crate source:
- delegate: states have no per-turn input override (only pipeline:/
  concurrent: stages get input: templates), so the critic just echoed
  the document back instead of answering yes/no.
- extract: context extractors read the state's incoming user_message,
  not its generated/delegated response (runtime.rs:7320), so a guard
  meant to gate on "what the critic just said" has nothing real to
  read -- the loop-back transition never fires.

Documented as a genuine finding in NOTES.md rather than forcing a
fragile demo: this crate's state machine is built for turn-based intent
routing, not gating on a sub-agent's structured verdict.
schaefera commented 2026-08-06 08:27:47 +00:00
Owner
Copy link

Rejected:

  • Project is maintained by one person and has no other commits from anybody who is not the author.
  • Real limitations in state flows found during spike, chalking up to lack of ecosystem maturity. Better alternatives will eventually appear.
Rejected: - Project is maintained by one person and has no other commits from anybody who is not the author. - Real limitations in state flows found during spike, chalking up to lack of ecosystem maturity. Better alternatives will eventually appear.
schaefera closed this pull request 2026-08-06 08:27:48 +00:00

Pull request closed

Please reopen this pull request to perform a merge.
Sign in to join this conversation.
Reviewers
No reviewers
Labels
Clear labels
No items
No labels
Milestone
Clear milestone
No items
No milestone
Projects
Clear projects
No items
No project
Assignees
Clear assignees
No assignees
2 participants
Notifications
Due date
The due date is invalid or out of range. Please use the format "yyyy-mm-dd".

No due date set.

Dependencies

No dependencies set.

Reference: schaefera/doubleo7#1
No description provided.
Delete branch "spike/ai-agents-yaml-config"

Deleting a branch is permanent. Although the deleted branch may continue to exist for a short time before it actually gets removed, it CANNOT be undone in most cases. Continue?