AI case study · 2026
AgentOps AI — Local-first RAG & Agent Platform
A production-minded RAG platform that runs entirely on local models: multi-tenant document ingestion, hybrid retrieval, and grounded answers with validated citations. The foundation is built; the agent harness on top of it is in progress.
107
tests passing
Hybrid
vector + keyword search
$0
local model cost
The problem
RAG demos tend to fall apart on real documents: versions contradict each other, vector search misses exact terms, one tenant's data can reach another's results, and a model can cite sources it never saw. Those problems have to be solved before an agent is handed tools.
What I built
I built a FastAPI service on PostgreSQL and pgvector that validates uploads, chunks by document structure, and indexes in one transaction so a document is never half-searchable. Search fuses vector and full-text results with reciprocal rank fusion, and every retriever builds on a single tenant-scoped query. Local Ollama models answer with JSON-schema output, retrieved text is fenced as untrusted, and citations are checked against the retrieved evidence.
Why it matters
Agents are only as trustworthy as the retrieval and isolation beneath them. Building those first means every tool added later inherits tenant scoping, cited evidence, and an honest answer when the documents don't contain one.
Results
- Ingestion for PDF, Markdown and text with magic-byte checks, content-hash idempotency and filename versioning, verified on a sample corpus with deliberately conflicting documents.
- Live end-to-end runs answered a v1-vs-v2 question citing both documents, returned insufficient_evidence for an out-of-scope question, and showed a second tenant 0 results.
- 107 unit and integration tests against real Postgres and pgvector, plus 3 live model tests, with Ruff and strict Mypy clean and CI on GitHub Actions.
- In progress: a policy-gated agent harness, a LangGraph workflow with human approval, evaluation, and drift monitoring.
