Case study · MILL5 · 2024 — 2026
Hybrid retrieval for more relevant RAG answers
Combining keyword and vector search, reranking and contextual chunking so the model sees the right evidence.
Staff AI Engineer — retrieval design
- Improvement in answer relevance
- 38%Improvement in answer relevance
- Earlier gain in retrieval relevance at WellSky
- 35%Earlier gain in retrieval relevance at WellSky
Context
Most of the answer-quality problems we saw in retrieval-augmented generation were not model problems. The model was answering well from the context it was given — the context was just the wrong context.
Approach
- Moved from pure vector search to hybrid search, combining lexical matching for exact terms such as names, codes and IDs with dense embeddings for meaning.
- Added a reranking stage so a small number of high-quality passages reached the model instead of a long tail of near-misses.
- Reworked chunking to keep surrounding context — section titles, document metadata — attached to each passage.
- Measured retrieval separately from generation, so we could tell whether a bad answer came from search or from the model.
Results
- Answer relevance improved by 38%.
- Debugging got faster because retrieval quality had its own metrics and its own regression tests.
What I took from it
This built on semantic search work at WellSky, where embeddings and vector databases improved information retrieval relevance by 35%. The lesson carried over: invest in retrieval first, then prompts, then models.
Stack
- Python
- LlamaIndex
- LangChain
- pgvector
- Pinecone
- Rerankers
- Embeddings