Skip to content

Case study · MILL5 · 2024 — 2026

Hybrid retrieval for more relevant RAG answers

Combining keyword and vector search, reranking and contextual chunking so the model sees the right evidence.

Staff AI Engineer — retrieval design

Improvement in answer relevance
38%Improvement in answer relevance
Earlier gain in retrieval relevance at WellSky
35%Earlier gain in retrieval relevance at WellSky

Context

Most of the answer-quality problems we saw in retrieval-augmented generation were not model problems. The model was answering well from the context it was given — the context was just the wrong context.

Approach

  • Moved from pure vector search to hybrid search, combining lexical matching for exact terms such as names, codes and IDs with dense embeddings for meaning.
  • Added a reranking stage so a small number of high-quality passages reached the model instead of a long tail of near-misses.
  • Reworked chunking to keep surrounding context — section titles, document metadata — attached to each passage.
  • Measured retrieval separately from generation, so we could tell whether a bad answer came from search or from the model.

Results

  • Answer relevance improved by 38%.
  • Debugging got faster because retrieval quality had its own metrics and its own regression tests.

What I took from it

This built on semantic search work at WellSky, where embeddings and vector databases improved information retrieval relevance by 35%. The lesson carried over: invest in retrieval first, then prompts, then models.

Stack

  • Python
  • LlamaIndex
  • LangChain
  • pgvector
  • Pinecone
  • Rerankers
  • Embeddings