RAG and Knowledge Systems

RAG and Knowledge Systems

RAG (retrieval-augmented generation) means: find the right documents first, then ask the model.

The model is only as good as the context you give it.

The pipeline

1. 1

   Chunk

   Split documents into pieces a model can actually use.

2. 2

   Embed / index

   Make those pieces searchable.

3. 3

   Retrieve

   Pull the top candidates for a query.

4. 4

   Rank

   Optional, but usually worth it — put the best snippets first.

5. 5

   Assemble + generate

   Build a prompt with citations, then generate an answer that stays inside that context.

Where RAG actually breaks

• chunks are too big (noise) or too small (no meaning)
• the query and the docs use different words
• you retrieve 20 snippets and dump all of them in
• you never check whether retrieval was relevant before blaming the model

The useful test

If a human could not answer from the retrieved snippets, the model should not be expected to either.

1. Print the retrieved chunks
2. Ask: would I answer from these alone?
3. If no → fix retrieval, not the prompt poem

Next

When retrieval is not enough and the system needs to do things:

• Agents and Tool Use