diff --git a/README.md b/README.md index 36d910e..82cd12a 100644 --- a/README.md +++ b/README.md @@ -27,7 +27,10 @@ Running the LLM on every article from every feed doesn't scale locally. Instead: 1. Every new article is embedded (`nomic-embed-text` by default) and scored against your preference-profile embedding — cheap, runs on all of them. 2. Only articles above `EMBEDDING_SHORTLIST_THRESHOLD` (`crates/web/src/server/pipeline.rs`) - go to the LLM (`llama3.1` by default) for an actual relevance judgment with a rationale. + go to the LLM (`gemma4-e4b` by default — chosen because it's already pulled locally and a + reasonable size/speed tradeoff for a call made once per shortlisted article on every + pipeline run; swap it in `crates/web/src/server.rs` for a bigger/smaller model as needed) + for an actual relevance judgment with a rationale. 3. `feedsignal_core::scoring::score_article` blends LLM score (dominant when present), embedding score (fallback / floor), and topic affinity (bounded nudge) into `final_score`, which is what the UI ranks by. @@ -82,7 +85,7 @@ Requires [Ollama](https://ollama.com) running locally with the models pulled: ```sh ollama pull nomic-embed-text -ollama pull llama3.1 +ollama pull gemma4-e4b ``` And the [Dioxus CLI](https://dioxuslabs.com/learn/0.7/getting_started/) (`dx`) to build/serve diff --git a/crates/web/src/server.rs b/crates/web/src/server.rs index ab5ea52..5291986 100644 --- a/crates/web/src/server.rs +++ b/crates/web/src/server.rs @@ -23,8 +23,8 @@ async fn ensure_background_jobs_started(db: Arc) { BACKGROUND_JOBS .get_or_init(|| async { // TODO: move base_url/model names to config/env once there's a - // settings story; hardcoded to the common local Ollama defaults. - let llm = Arc::new(Llm::new("http://localhost:11434", "nomic-embed-text", 768, "llama3.1").expect("failed to construct ollama client")); + // settings story; hardcoded to models already pulled locally. + let llm = Arc::new(Llm::new("http://localhost:11434", "nomic-embed-text", 768, "gemma4-e4b").expect("failed to construct ollama client")); if let Err(err) = pipeline::start_scheduler(db, llm).await { tracing::error!(?err, "failed to start background job scheduler"); }