feedsignal/README.md

# feedsignal

A self-hosted RSS reader that uses a local Ollama LLM (via [rig](https://github.com/0xPlaygrounds/rig))
to surface the articles actually worth reading out of a large set of subscribed feeds, and learns
from what you read (and how long, and what you dismiss) to get better at that over time.

## Architecture

```
crates/
  core/    domain models (Article, Feed, ReadingEvent) + the topic-affinity
           learning engine + the final relevance-scoring formula. No I/O.
  db/      SQLite (sqlx) persistence: feeds, articles, the append-only
           reading_events log, topic_affinities, preferences.
  feeds/   RSS/Atom fetching + parsing (feed-rs).
  llm/     rig + local Ollama: embeddings for the cheap first-pass filter,
           chat completion for the relevance judgment on the shortlist.
  web/     axum + Dioxus fullstack UI. `server` feature = native binary
           (DB, feed polling, LLM calls, scheduler); `web` feature = the
           WASM client shipped to the browser. No separate JS/npm stack.
```

### Two-stage relevance filtering

Running the LLM on every article from every feed doesn't scale locally. Instead:

1. Every new article is embedded (`nomic-embed-text` by default) and scored against your
   preference-profile embedding — cheap, runs on all of them.
2. Only articles above `EMBEDDING_SHORTLIST_THRESHOLD` (`crates/web/src/server/pipeline.rs`)
   go to the LLM (`gemma4-e4b` by default — chosen because it's already pulled locally and a
   reasonable size/speed tradeoff for a call made once per shortlisted article on every
   pipeline run; swap it in `crates/web/src/server.rs` for a bigger/smaller model as needed)
   for an actual relevance judgment with a rationale.
3. `feedsignal_core::scoring::score_article` blends LLM score (dominant when present),
   embedding score (fallback / floor), and topic affinity (bounded nudge) into `final_score`,
   which is what the UI ranks by.

### Learning from reading behavior

Every interaction (impression, open, dwell time, star, dismiss) is appended to an immutable
`reading_events` log — never mutated, so the model can be recomputed or retuned later without
losing data (`crates/db/migrations/0001_init.sql`).

From that log, `feedsignal_core::affinity`:

- Computes an **engagement score** per article (0.0–1.0) from whether it was opened, how long
  you spent relative to its estimated read time, and explicit star/dismiss signals.
- Compares engagement to what the pipeline *predicted* (`final_score` at ingest time) to get a
  **surprise** signal — did you engage more or less than expected?
- Nudges the affinity of that article's topics by `learning_rate * surprise`, clamped to
  `[-1, 1]`, so "the model rated this low but I read the whole thing" measurably shifts future
  scoring, and "the model rated this high but I dismissed it unread" pulls the other way.
- Decays all affinities toward zero once a day so stale interests fade instead of anchoring
  the model forever.

This is a from-scratch numeric signal engine, deliberately not an LLM-rewritten prose "taste
profile" — the topic scores are inspectable, and the update rule is simple enough to reason
about and tune. See `crates/core/src/affinity.rs` for the full implementation and unit tests.

## Status

This is a scaffold, not a working app yet. What's real and compiles (`cargo check`/`cargo test`
pass for every crate, both the `server` and `wasm32-unknown-unknown` targets of `feedsignal-web`):

- The domain model, DB schema/migrations, and the topic-affinity engine (with tests).
- Feed fetching/parsing.
- Ollama embedding + chat-completion calls via rig (against real rig-core 0.42 APIs).
- The scoring pipeline shape and a cron-scheduled job wiring (`tokio-cron-scheduler`).
- A minimal Dioxus UI that lists ranked articles and lets you dismiss one.

What's *not* wired up yet (left as the natural next steps):

- Feed subscription management (add/remove feed URLs) — `feedsignal_feeds::fetch_feed` exists
  but nothing calls it on a schedule yet.
- The `preferences` table (your free-text "what I care about" description) isn't read by the
  pipeline yet — `pipeline.rs` has a `TODO` where it belongs.
- Dwell-time capture from the browser (needs a small bit of JS/visibility-API glue on the
  article view, or a "mark read" action, to backfill `dwell_seconds` on the `opened` event).
- Full article content extraction (currently only the feed's summary/content field is used;
  no readability-style scraping of the linked page).

## Running it

Requires [Ollama](https://ollama.com) running locally with the models pulled:

```sh
ollama pull nomic-embed-text
ollama pull gemma4-e4b
```

And the [Dioxus CLI](https://dioxuslabs.com/learn/0.7/getting_started/) (`dx`) to build/serve
the fullstack app — `dx` also has an `dx add`/component-scaffolding workflow worth using instead
of hand-rolling UI pieces as this grows:

```sh
cargo install dioxus-cli
cd crates/web
dx serve
```

The native crates (`core`, `db`, `feeds`, `llm`) build with plain `cargo check`/`cargo test` and
don't need `dx` at all.

## CI

`.forgejo/workflows/ci.yml` runs on the self-hosted Forgejo runner: checks the native crates,
both feature-sets of `feedsignal-web` (server + wasm client), and runs tests.