Groundwork
Solo buildA fully offline LLM application with a deterministic verification layer. Runs open-source models locally. No paid APIs, no external calls.
Working application, solo-scoped and shipped. Runs on consumer hardware with zero API costs. Proves that open-source models can produce reliable output when verification is designed into the system.
How It Works
Key Product Decisions
Fully local, fully offline
Only network traffic is to localhost (Ollama). No paid APIs, no external hosts. Privacy and per-query cost are both zero.
Deterministic verification
A quoted span is surfaced only if it is a normalized verbatim substring (25+ chars or 6+ words) of the cited source record. No model involved. Pure string matching. Fails are flagged, never shown.
Defensive output handling
Strips reasoning blocks, extracts first balanced JSON object (string/escape aware), retries JSON-only, normalizes fields, classifies timeouts as ERROR (never conflated with verification failure).
Immutable audit trail
Raw model responses stored before parsing. Every verdict references them. Every displayed metric traces to a stored row.
Test infrastructure included
pytest suite covering model availability, ingest/embed/retrieve, generation parsing, verifier unit cases, and a retrieval eval harness. All local, all offline.
The Story
The core question: can you get reliable, verifiable output from open-source models running on consumer hardware? Yes, if you design the right verification layer.
The system runs entirely on the user's machine via Ollama. Embeddings are generated and cached locally in SQLite. The deterministic verification layer is the key trust decision - model output is never shown to the user unless it can be mechanically confirmed against source records. No model is involved in that confirmation.
Video walkthrough coming soon.