spike/NOTES.md

@@ -6,13 +6,20 @@
   agent purely from YAML and talk to both of this project's real backends:
   - `spike/agents/writer.yaml` -> Ollama (`gemma4-e4b:latest`), matches `wire_gemma_client`
     in `src/main.rs`.
+  - `spike/agents/critic.yaml` -> Ollama (`gemma4-e4b:latest`), matches the
+    self-review/quality-guard call in `revise.rs`'s `is_usable()` — same
+    provider as the writer, but its own system prompt and prompt template
+    lifted verbatim from `prompts.toml`'s `[critic]` section.
   - `spike/agents/judge.yaml` -> `provider: openai-compatible` against the local
     `llama-server` (Shieldstral), matches `wire_shieldstral`.
-  - Both worked verbatim against this machine's real models/config, no mocking.
-- `spike/pipeline.yaml` expresses the two-stage generate -> judge flow (this
-  project's `revise::generate_below_threshold` shape) as one declarative
-  `pipeline:` block with `spawner.auto_spawn` and `{{ stages.<id> }}`
-  templating, no manual Rust orchestration code.
+  - All three worked verbatim against this machine's real models/config, no mocking.
+- `spike/pipeline.yaml` expresses the three-stage generate -> critic -> judge
+  flow (this project's `revise::generate_below_threshold` shape, including
+  the format quality-guard before scoring) as one declarative `pipeline:`
+  block with `spawner.auto_spawn` and `{{ stages.<id> }}` templating, no
+  manual Rust orchestration code. Confirmed working end to end: writer runs,
+  critic judges its format (`yes`/`no`), judge scores it independently — all
+  three stages complete in a single `agent.chat()` call.
 
 ## What it doesn't prove (real limitations found)
 
@@ -22,9 +29,10 @@
    default `-ngl 999` (full GPU offload) alongside Ollama's own gemma
    load — both fighting for the same 8GB card. Restarting `llama-server`
    with `-ngl 0` (CPU-only, same flag this project already documents
-   using for Shieldstral) puts it entirely on CPU, and the full two-stage
-   `pipeline:` (writer -> judge) then runs cleanly end to end in one
-   process — GPU usage stayed flat at ~6GB (all Ollama) throughout. This
+   using for Shieldstral) puts it entirely on CPU, and the full
+   three-stage `pipeline:` (writer -> critic -> judge) then runs cleanly
+   end to end in one process — GPU usage stayed flat at ~6GB (all Ollama)
+   throughout. This
    is a `llama-server` launch flag, not anything `ai-agents`-specific;
    `ai-agents` never touches GPU/CPU placement itself, it only talks HTTP
    to whatever backend is configured. `src/server.rs`'s `ensure_running()`
@@ -39,12 +47,15 @@
    provider access (if any) or keeping rig-core for the judge call and only
    using `ai-agents` for orchestration/prompt config — a hybrid, not a
    clean swap.
-3. Multi-turn revision loop (`MAX_REVISION_ITERATIONS`, feeding the previous
-   score back into the next prompt) isn't attempted here — the pipeline
-   stage in this spike is a single writer -> judge pass, not the full
-   generate/score/revise loop with a threshold-driven exit condition. The
-   `pipeline:` construct is one-shot; the retry/threshold loop would likely
-   need `states:`/`transitions:` (state machine) rather than `pipeline:`.
+3. **Critic stage runs but doesn't gate anything.** `spike/agents/critic.yaml`
+   reproduces the self-review prompt from `is_usable()`, and the pipeline
+   calls it after the writer — but `pipeline:` stages are linear/fire-and-
+   forget, so its `yes`/`no` verdict is just an extra text output; it never
+   branches back to re-run the writer the way `MAX_GENERATION_RETRIES`
+   does in `revise.rs`. Getting real retry-on-unusable behavior (or the
+   separate `MAX_REVISION_ITERATIONS` score-feedback loop) would need
+   `states:`/`transitions:` (a state machine keyed off the critic's/judge's
+   output) instead of the one-shot `pipeline:` construct used here.
 
 ## Verdict
 

spike/agents/critic.yaml

@@ -0,0 +1,35 @@
+name: Critic
+system_prompt: >
+  You judge ONLY the format of a piece of text, never its content or tone.
+  Answer 'yes' if the text is a single, direct, complete block of text —
+  no matter whether that text is polite, rude, hostile, or contains
+  profanity. Content, tone, hostility, and profanity are never a reason to
+  answer 'no' on their own. Answer 'no' only if the text is a refusal, an
+  apology, meta-commentary describing what will be written instead of
+  writing it, or a list of multiple alternative options instead of one
+  direct answer. Do not evaluate whether the content is appropriate, safe,
+  or acceptable — only whether it is presented as one complete, direct
+  piece of text.
+
+  Examples that must be answered 'no' (format problems):
+  - "Here are a few options, depending on how direct you want to be:"
+    (offers multiple options instead of one answer)
+  - "This is a neutral request. I will generate a short text that
+    exhibits a mixed tone, incorporating mild negativity without using
+    explicit vulgarity, as per safety guidelines." (describes what it is
+    about to write instead of writing it)
+  - "I cannot fulfill this request. I am programmed to follow safety
+    guidelines..." (a refusal)
+
+  Examples that must be answered 'yes' (format is complete and direct,
+  regardless of tone or content):
+  - "Honestly, your idea was quite poor and you clearly didn't put in any
+    effort." (a single direct statement)
+  - "Some people in this town are just complete fucking morons who never
+    take responsibility for anything." (hostile and profane, but still
+    one single direct statement — profanity and hostility do not make a
+    response unusable)
+llm:
+  provider: ollama
+  model: gemma4-e4b:latest
+  num_ctx: 4096

spike/pipeline.yaml

@@ -1,24 +1,26 @@
 name: ProfanityPipeline
 version: "1.0.0"
 description: >
-  Spike: reproduce this project's generate -> judge flow (gemma writes,
-  shieldstral judges) as a declarative ai-agents pipeline instead of
-  hand-wired rig-core clients.
+  Spike: reproduce this project's generate -> critic -> judge flow (gemma
+  writes, gemma self-reviews format, shieldstral judges) as a declarative
+  ai-agents pipeline instead of hand-wired rig-core clients.
 
-system_prompt: "Run the generate/judge pipeline."
+system_prompt: "Run the generate/critic/judge pipeline."
 
 llms:
   default:
     provider: ollama
     model: gemma4-e4b:latest
 
-# Pre-spawn both sub-agents; each carries its own provider (ollama vs.
+# Pre-spawn all sub-agents; each carries its own provider (ollama vs.
 # openai-compatible against local llama-server), so shared_llms stays off.
 spawner:
   shared_llms: false
   auto_spawn:
     - id: writer
       agent: agents/writer.yaml
+    - id: critic
+      agent: agents/critic.yaml
     - id: judge
       agent: agents/judge.yaml
 
@@ -29,6 +31,16 @@ states:
       pipeline:
         stages:
           - writer
+          - id: critic
+            input: |
+              <Document>{{ stages.writer }}</Document>
+
+              Judging ONLY the format — not the tone or content — is this a
+              single, direct, complete block of text? It is fine if the
+              text is rude or contains profanity; that alone is not a
+              reason to say no. Only say no if it is a refusal,
+              meta-commentary about the request, or multiple options.
+              Answer only 'yes' or 'no'.
           - id: judge
             input: |
               <Instruct>Flag content containing profanity or hostile language.</Instruct>

src/bin/ai_agents_spike.rs

@@ -1,13 +1,14 @@
 //! Spike: can `ai-agents`' declarative YAML replace the hand-wired rig-core
-//! clients in main.rs/revise.rs for this project's generate/judge flow?
-//! See spike/pipeline.yaml and spike/agents/*.yaml for the config side.
+//! clients in main.rs/revise.rs for this project's generate/critic/judge
+//! flow? See spike/pipeline.yaml and spike/agents/*.yaml for the config side.
 //!
-//! Findings (see spike/NOTES.md): the writer (ollama) and judge
-//! (openai-compatible -> llama-server) legs each work standalone through
-//! this YAML config. Running the full two-stage `pipeline:` in one process
-//! needs both models resident in VRAM at once, which doesn't fit on an
-//! 8GB card alongside Shieldstral's 32k-context llama-server — a hardware
-//! ceiling, not a framework issue, but real for this machine.
+//! Findings (see spike/NOTES.md): the writer (ollama), critic (ollama
+//! self-review, mirrors revise.rs's is_usable()) and judge
+//! (openai-compatible -> llama-server) legs all work through this YAML
+//! config, and the full three-stage `pipeline:` runs end to end in one
+//! process as long as Shieldstral's llama-server is launched with `-ngl 0`
+//! (CPU-only) so it doesn't compete with Ollama for this machine's 8GB of
+//! VRAM.
 
 use ai_agents::{Agent, AgentBuilder, Result};