Default LLM scoring stage to a model already pulled locally
Some checks failed
CI / check (push) Failing after 11s

Was hardcoded to llama3.1, which isn't actually on this machine.
Switched to gemma4-e4b (5.0 GB, already pulled) as a speed/quality
balance for a call made once per shortlisted article on every
pipeline run.
This commit is contained in:
Austin Schaefer 2026-08-20 17:11:33 +02:00
parent 1a4a72fc27
commit 2b61c80143
2 changed files with 7 additions and 4 deletions

View file

@ -27,7 +27,10 @@ Running the LLM on every article from every feed doesn't scale locally. Instead:
1. Every new article is embedded (`nomic-embed-text` by default) and scored against your
preference-profile embedding — cheap, runs on all of them.
2. Only articles above `EMBEDDING_SHORTLIST_THRESHOLD` (`crates/web/src/server/pipeline.rs`)
go to the LLM (`llama3.1` by default) for an actual relevance judgment with a rationale.
go to the LLM (`gemma4-e4b` by default — chosen because it's already pulled locally and a
reasonable size/speed tradeoff for a call made once per shortlisted article on every
pipeline run; swap it in `crates/web/src/server.rs` for a bigger/smaller model as needed)
for an actual relevance judgment with a rationale.
3. `feedsignal_core::scoring::score_article` blends LLM score (dominant when present),
embedding score (fallback / floor), and topic affinity (bounded nudge) into `final_score`,
which is what the UI ranks by.
@ -82,7 +85,7 @@ Requires [Ollama](https://ollama.com) running locally with the models pulled:
```sh
ollama pull nomic-embed-text
ollama pull llama3.1
ollama pull gemma4-e4b
```
And the [Dioxus CLI](https://dioxuslabs.com/learn/0.7/getting_started/) (`dx`) to build/serve

View file

@ -23,8 +23,8 @@ async fn ensure_background_jobs_started(db: Arc<Db>) {
BACKGROUND_JOBS
.get_or_init(|| async {
// TODO: move base_url/model names to config/env once there's a
// settings story; hardcoded to the common local Ollama defaults.
let llm = Arc::new(Llm::new("http://localhost:11434", "nomic-embed-text", 768, "llama3.1").expect("failed to construct ollama client"));
// settings story; hardcoded to models already pulled locally.
let llm = Arc::new(Llm::new("http://localhost:11434", "nomic-embed-text", 768, "gemma4-e4b").expect("failed to construct ollama client"));
if let Err(err) = pipeline::start_scheduler(db, llm).await {
tracing::error!(?err, "failed to start background job scheduler");
}