Cosimi: a portfolio chatbot that answers without an LLM

This chat is powered by Cosimi, a deterministic retrieval engine and SDK I built - GraphRAG with no LLM in the request path. Your question is embedded, a vector search seeds from the nearest chunks and pairs, the engine walks a graph for context, ranks, and returns - all deterministic, against PostgreSQL with pgvector. Every answer is source-traceable: the corpus it answers from is grounded on documents I wrote. The system either has an answer it built from my source, or it tells you it doesn't.

Why a chatbot without a live LLM

The default move for a portfolio chat is to bolt a hosted LLM onto a "talk to me" surface. That solves the dialogue problem but creates a worse one - the replies are no longer mine, they're the model's prior over what a developer named Long might plausibly say, smoothed over with corporate-safe verbs. If a recruiter asks me something I haven't thought through, I'd rather the answer be silence than a confident hallucination in my voice. Keeping generation out of the live path makes the failure mode legible: either there's a grounded reply, or there isn't.

The build-time pipeline

The LLM does almost all the labor, but offline, before anyone is watching. The ingest pipeline runs on my machine: it takes my source documents, chunks them into a graph, generates question-answer pairs with one Claude model, audits every pair with a second model, and embeds the survivors into Postgres with pgvector (bge-m3, 1024-dim, HNSW index). Chunks and pairs are model-generated, none hand-typed. Then it freezes. The model that wrote the answers never sees the question that retrieves them.

Retrieval

At request time there is no model. The query is embedded; a vector search seeds from the nearest chunks and pairs by cosine similarity; the engine walks the chunk-relation graph for surrounding context; it ranks and returns the best answer. Same question, same data, same answer, every time, at zero per-reply token cost on the edge. A deterministic deflect guard screens abusive or overly personal inputs before anything is embedded, and a curated layer of hand-written seed Q&A - chip labels and common openers - matches by embedding ahead of the graph retriever; the long tail falls through to the retriever.

Trade-offs and implementation

The trade is depth-for-honesty: every reply is grounded on source I wrote and gated by an audit, not improvised live. What I own isn't the generated corpus - it's the source it's grounded on, the audit gate, and the curated seed layer. The build is a pnpm + Turborepo monorepo on Node 22, tested with Vitest. The retrieval runtime is packaged as a reusable TypeScript SDK with injected SQL and embedder dependencies, so the same engine runs anywhere I can hand it a postgres.js client and an embedder. The public API is a Hono server on Cloudflare Workers, connected through Hyperdrive to self-hosted PostgreSQL 16 with pgvector, with no LLM in its dependency graph. Streaming replies go out over SSE, paced so the reply feels typed - the only part that pretends to be live. The cost is real: long-tail inputs miss and the corpus needs maintenance. For a portfolio chat that speaks as me, that's the right trade.

The engine ships as a set of @cosimi/* packages on npm, and the code is at github.com/8bu/cosimi.