RAG: the model doesn't have to know everything by heart

Models have knowledge frozen at training. everything that happened after, or everything internal to your organization, they don't know. Two ways to give them access:

1. Tools (Track 3). the model invokes functions that fetch info in real time.
2. RAG. before generating, you fetch relevant data and inject it into the prompt as context.

RAG is the dominant choice when:

• You have a large corpus (manuals, docs, transcripts).
• Questions are about content, not actions.
• You want answers to come with citations (verifiable).

The three stages

[user query]
        ↓
   retrieve   (vector search over the corpus → top-K snippets)
        ↓
   augment    (composes a prompt: snippets + original question)
        ↓
   generate   (LLM answers using the augmented prompt, citing sources)
        ↓
[answer + citations]

Your job: design the YAML config of the pipeline. Three steps, chained by output_keys.

The trap that breaks RAG

The most common problem isn't retrieval. it's that the generate doesn't handle the "snippets aren't relevant to the question" case. When that happens, if you don't tell the model what to do, it invents an answer with bits from the snippets. That's worse than not answering: it looks confident, carries fake citations, and the user doesn't notice.

Practical rule: in the generator's prompt, always include something like "If the snippets don't contain relevant information, say you didn't find that in the manual." It's the difference between honest RAG and hallucinating RAG.

How it's evaluated

5 LLM-judge criteria:

1. retrieve declared as a tool/function call (not LLM).
2. augment receives question + snippets, combines them explicitly.
3. generate uses the augmented_prompt (not the raw question).
4. generate asks for citations from the snippets.
5. generate handles "empty / not-relevant snippets" without hallucinating.