RAG — search by meaning
About this app
Retrieval-augmented generation step by step: chunks, embeddings, search by cosine, BM25 or hybrid, top-k cut-off, prompt with token budget and an answer with sources; missed chunks lead to hallucination.
How RAG works
Retrieval-augmented generation (RAG) lets a language model look things up before it answers. Documents are split into pieces called chunks, and each chunk gets an embedding, a vector of its meaning. The question is embedded too. Retrieval takes the chunks with the highest cosine similarity up to a top-k cut-off and copies them into the prompt together with the question. The model answers from that context and can cite its sources. If the right chunk is missing from the prompt, it answers from its training data and may hallucinate.
What the app shows
The app shows the five steps chunk, embed, retrieve, augment and generate using a help centre as the example. Chunks sit as points on a sphere, grouped by topics such as contract, payment, technology and data protection. A ranking lists all chunks with cosine and BM25 scores, and a line marks the top-k cut. On the right a 256-token context window fills up. The embeddings are hand-made toy vectors with 16 concept axes, and the answer is assembled from sentences in the prompt rather than produced by a real language model.
Things to try
Try the example questions, which cover synonyms, ambiguity, error codes and edge cases, or type your own. Compare the search modes: keyword search (BM25) cannot match “vehicle” when you ask about a “car”, but it hits exact codes such as E-217; semantic search behaves the other way round, and hybrid merges both rankings. Set top-k between 1 and 6 and the chunk size to sentence, paragraph or document, and watch when the answer is grounded, incomplete or hallucinated.
Frequently asked questions
- What is RAG in AI?
- RAG stands for retrieval-augmented generation. Before answering, a language model receives relevant passages from a document collection in its prompt, selected by a search step. This lets it answer questions about content that is not in its training data and cite the sources.
- What is the difference between semantic search and BM25?
- BM25 is a classic keyword method that scores chunks by the words they share with the question. It does not find synonyms but reliably matches exact terms and codes. Semantic search compares embeddings by cosine similarity and finds passages with the same meaning, but can miss exact identifiers. Hybrid search combines both rankings.
- Why does a RAG system still hallucinate?
- A RAG system hallucinates when the chunk containing the answer does not end up in the prompt. That happens when retrieval does not find it, when it ranks below the top-k cut, or when it no longer fits into the token budget. The model then fills the gap with plausible knowledge from training.
- What can I try in this app?
- You ask example or custom questions and choose keyword, semantic or hybrid search, top-k and chunk size. You see the ranking, the assembled prompt with its token budget, and whether the answer is backed by sources.
Subject: Neural networks
More from Neural networks
- Agent loop
- Attention Visualiser
- Backpropagation — blame in reverse
- Backpropagation: learning from one mistake
- Basin of Attraction
- Your job under AI
- The temperature dial
- Diffusion model — images from noise (3D)
A VisuApp by heyprof: interactive, free in your browser, no sign-up. What is a VisuApp? · All apps