doubleo7/README.md
Austin Schaefer f2c10783db
Some checks failed
CI / test (push) Failing after 6s
CI / test (pull_request) Failing after 7s
Extract swear_cleanup to its own repo, flatten deep_research to root
deep_research is the only project this repo is meant to showcase, so the
Cargo workspace wrapping it and an unrelated side project no longer earns
its keep:

- swear_cleanup moved to a new standalone local repo (~/dev/swear_cleanup,
  not pushed anywhere) via `git subtree split`, with its pre-workspace-
  split history (when it lived at src/swear_cleanup/ in a single shared
  crate) spliced onto its post-split history rather than starting from a
  single flattened snapshot. FINDINGS.md, which was sitting at this repo's
  root but was actually swear_cleanup's own build log, went with it.
- deep_research/{src,Cargo.toml,README.md,docs} moved to the repo root;
  the [workspace] table collapsed into a plain [package] manifest with
  dependency versions inlined from the old [workspace.dependencies].
- Cargo.toml keeps an explicit empty [workspace] table (not just omitted)
  so that checking this repo out as a nested git worktree — this
  project's own normal workflow — can't accidentally inherit a stale
  ancestor directory's workspace manifest, which is exactly what broke
  the build while testing this change from a worktree.
- .forgejo/workflows/deep_research-ci.yml -> ci.yml, dropping the now-
  meaningless -p deep_research scoping and path filters (redundant when
  it's the only thing in the repo).
- README.md and docs/case-study.md updated for the flattened commands
  (cargo run/test with no -p flag); their relative links to each other
  and to src/ were already correct since both moved together.

Verified: cargo build/test/clippy/fmt all clean from the new repo root.
2026-08-18 13:43:26 +02:00

4.9 KiB
Raw Blame History

deep_research

A local-first, multi-agent deep-research CLI: give it a topic, it searches the web, cross-checks what it finds, and writes up a cited report — entirely on infrastructure you control, with no cloud LLM API key and no query ever leaving your machine.

$ deep_research "trends in AI customer-support chatbots"
🔎 Researching...
🧐 Reviewing findings...
✍️  Writing report...

# AI Customer-Support Chatbots: 20252026 Trends
...

Why this exists

This started as a "does deep research actually work end-to-end" exercise and turned into a small case study in building an agentic system that survives contact with reality: models that hit their turn budget mid-task, search providers that rate-limit, and reviewers that reject good-faith work. The case study walks through what broke and how each failure was fixed, not just papered over.

Architecture

Four small agents, each with one job, coordinated by plain Rust control flow — not a framework's agent graph, not an LLM deciding when to stop:

┌─────────────┐  approve/reject  ┌──────────┐
│  researcher │ ───────────────► │ reviewer │
│ (tool-using)│ ◄─────────────── │          │
└──────┬──────┘   gaps/feedback  └────┬─────┘
       │ turn budget exhausted        │ approved,
       │ mid-investigation            │ or out of rounds
       ▼                              ▼
┌──────────────┐              ┌──────────────┐
│ summarizer   │──findings───►│    writer    │──► report
│ (recovery)   │              │              │
└──────────────┘              └──────────────┘
  • researcher — a tool-calling agent (search_web, fetch_page) that gathers and cross-checks evidence, capped at a fixed model-call budget so a confused model can't loop forever.
  • reviewer — a separate, fresh-context agent that checks the researcher's conclusions actually follow from its cited sources, and either approves the findings or hands back concrete gaps for another pass.
  • writer — turns approved (or partial) findings into a structured, footnoted report, streamed to the terminal as it's generated.
  • summarizer (recovery path) — only runs when the researcher exhausts its turn budget before concluding on its own. It reconstructs a proper findings dump from the raw tool-call transcript rather than the run simply failing; see the case study for why this exists and how it degrades gracefully if the summarizer call itself fails.

Everything runs against local models via Ollama and a self-hosted SearXNG instance for search — no OpenAI/ Anthropic/Google API key, no third-party search API, nothing about the research topic leaves the host it runs on. That's a deliberate constraint, not a limitation: it's the same shape a privacy-sensitive customer deployment would need.

Running it

Prerequisites:

  • Ollama running locally with a tool-calling-capable model pulled (the researcher and reviewer/writer models are configured in src/models.rs)
  • A local SearXNG instance with its JSON API enabled (defaults to http://localhost:8080, overridable via SEARXNG_URL)
cargo run -- "your research topic"

# or, with tracing spans on stderr instead of the progress spinner:
cargo run -- -l info "your research topic"

Project layout

Split one concern per file rather than one large module:

File Responsibility
main.rs Argument parsing, logging setup, and the single top-level call — no orchestration logic
research.rs The research/review round loop
researcher.rs The tool-calling research phase
review.rs The reviewer agent
summarizer.rs Max-turns recovery: reconstructs findings via a model call
writer.rs Turns findings into the final streamed report
history.rs Pure, unit-tested helpers for parsing a rig chat history into usable text
tools.rs search_web (SearXNG) and fetch_page tool implementations
stream.rs Drains a streaming prompt response to the terminal
progress.rs The terminal spinner and per-phase emoji
models.rs, cli.rs, observability.rs Small shared config: model names, CLI args, tracing setup

Testing

cargo test               # unit tests — pure functions, no network
cargo test -- --ignored  # + a live smoke test against SearXNG
cargo clippy --all-targets

CI (.forgejo/workflows/ci.yml) runs formatting, lint, build, and the unit test suite on every push and PR.