- retrieval::retrieve_relevant now shows a spinner around the query-
embedding call and gets its own tracing span, matching every other
model-calling phase in the pipeline (it previously ran invisibly and
untraced)
- add progress::report() to dedupe the show_progress-gated eprintln!
pattern shared by research.rs and retrieval.rs
- documents::push_document builds its TextSplitter once per collect_documents
call instead of once per file, skips the char-truncation walk entirely
when a file is already under the limit, and pulls the source-string
branch out of the map closure
- researcher::gather_findings drops a doc_context emptiness check that
build_doc_context already guarantees
- documents.rs tests use tempfile::tempdir() instead of a hand-rolled
TempDir type
Verified with cargo build/test/clippy -D warnings/fmt --check.
Adds --doc (repeatable, file or directory) so the researcher can draw on
user-supplied documents alongside the web: documents.rs resolves paths
into embeddable text, retrieval.rs embeds them with a dedicated
embedding model (nomic-embed-text, separate from the chat models used
elsewhere) into an in-memory vector index and retrieves the excerpts
most relevant to the topic once up front, and researcher.rs folds those
excerpts into the researcher's task under the same footnote-citation
scheme already used for web sources. The embedding and retrieval phases
show progress the same way every other phase does — a spinner while
embedding, a summary line once excerpts are retrieved, tracing spans
for -l mode.
Verified against a live Ollama nomic-embed-text pull and a real
research round: a planted fact sheet was correctly ranked as the most
relevant of several embedded documents and appeared in the researcher's
task before its first turn.
deep_research is the only project this repo is meant to showcase, so the
Cargo workspace wrapping it and an unrelated side project no longer earns
its keep:
- swear_cleanup moved to a new standalone local repo (~/dev/swear_cleanup,
not pushed anywhere) via `git subtree split`, with its pre-workspace-
split history (when it lived at src/swear_cleanup/ in a single shared
crate) spliced onto its post-split history rather than starting from a
single flattened snapshot. FINDINGS.md, which was sitting at this repo's
root but was actually swear_cleanup's own build log, went with it.
- deep_research/{src,Cargo.toml,README.md,docs} moved to the repo root;
the [workspace] table collapsed into a plain [package] manifest with
dependency versions inlined from the old [workspace.dependencies].
- Cargo.toml keeps an explicit empty [workspace] table (not just omitted)
so that checking this repo out as a nested git worktree — this
project's own normal workflow — can't accidentally inherit a stale
ancestor directory's workspace manifest, which is exactly what broke
the build while testing this change from a worktree.
- .forgejo/workflows/deep_research-ci.yml -> ci.yml, dropping the now-
meaningless -p deep_research scoping and path filters (redundant when
it's the only thing in the repo).
- README.md and docs/case-study.md updated for the flattened commands
(cargo run/test with no -p flag); their relative links to each other
and to src/ were already correct since both moved together.
Verified: cargo build/test/clippy/fmt all clean from the new repo root.
2026-08-18 13:43:26 +02:00
Renamed from deep_research/src/researcher.rs (Browse further)
