M7-B — How RAG Works Behind the Scenes
You do not need to code RAG to understand it. This note walks through what the computer does after you click "Upload PDF" — step by step, like a factory tour.
🏭 The full RAG factory line
STEP 1 Upload document
STEP 2 Cut into chunks (small pieces)
STEP 3 Convert each chunk to numbers (embeddings)
STEP 4 Store in a special search index (vector store)
─── setup done ───
STEP 5 You ask a question
STEP 6 Question also becomes numbers
STEP 7 Find chunks with similar numbers
STEP 8 Paste best chunks + question into AI
STEP 9 AI writes answer
That is the whole backend story.
✂️ Step 2 — Why cut into chunks?
AI cannot swallow a 500-page book in one bite. It has a context window (Module 2).
So software splits files into chunks — usually a few paragraphs each.
Whole textbook
↓
Chunk 1: "Chapter 1 — What is AI"
Chunk 2: "Chapter 1 — Machine Learning basics"
Chunk 3: "Chapter 2 — Prompting"
...
| Chunk too big | Chunk too small |
|---|---|
| Might not fit in AI memory | Loses context ("it" = what?) |
| Retrieval less precise | Too many fragments |
Good systems pick a middle size automatically.
🔢 Step 3 — What are embeddings?
An embedding turns text into a list of numbers that capture meaning.
Not spelling. Meaning.
Library analogy
Books on the same topic sit near each other on shelves:
"dog training" → shelf A
"puppy behaviour" → shelf A (close!)
"car engine repair"→ shelf Z (far away)
Embeddings do this in math space — thousands of dimensions you cannot draw, but the computer can measure distance.
| Sentence A | Sentence B | Distance |
|---|---|---|
| "How do LLMs work?" | "Explain large language models" | Small → similar |
| "How do LLMs work?" | "Best biryani in Mumbai" | Huge → not similar |
Search becomes "find meaning-nearby chunks" — not just keyword match like Ctrl+F.
🗄️ Step 4 — Vector database (simple name)
A vector database stores embeddings and finds nearest neighbours fast.
| Normal database | Vector database |
|---|---|
| "Find row where name = Raj" | "Find paragraphs most similar to this question" |
| Exact match | Meaning match |
| Excel-style tables | Math-space shelves |
You do not need to pick one yet. Tools like NotebookLM hide this inside the product.
🔍 Steps 5–8 — Question time
Question: "What is zero-shot prompting?"
Question → embedding [0.2, -0.8, 0.5, ...]
↓
Compare with all chunk embeddings
↓
Top 3–5 chunks win
↓
AI prompt becomes:
"Using ONLY this context:
[paste chunk 1]
[paste chunk 2]
Answer: What is zero-shot prompting?"
The AI never saw your whole book — only the winning chunks.
That is why a clear question gets better answers.
🖥️ Backend diagram (all parts together)
┌─────────────┐ ┌──────────────┐ ┌─────────────┐
│ Frontend │ │ Backend │ │ AI Model │
│ (website │────▶│ (chunks + │────▶│ (ChatGPT / │
│ or app) │ │ search) │ │ Gemini) │
└─────────────┘ └──────────────┘ └─────────────┘
│ │
│ ▼
│ ┌──────────────┐
└───────────▶│ Vector store │
│ (embeddings) │
└──────────────┘
| Part | Plain English |
|---|---|
| Frontend | Screen where you type and upload |
| Backend | Server that chops files and runs search |
| Vector store | Filing cabinet of meaning-numbers |
| AI model | Writes the final English answer |
Assignment (Assignment) Solve Questions asks you to explain this flow in a book-style answer — use this note as your outline.
🆚 RAG vs fine-tuning (don't mix them up)
| RAG | Fine-tuning |
|---|---|
| Add documents at runtime | Retrain model weights (expensive) |
| Change docs anytime | Hard to update |
| Good for facts & manuals | Good for style/behaviour |
| What this course focuses on | Advanced — later |
✅ Quick self-test
- What is a chunk?
- What is an embedding in one sentence?
- Why use vector search instead of Ctrl+F?
- What gets sent to the AI — whole PDF or top chunks?
If you can answer these, you understand RAG backend better than most beginners.
Previous: M7-A — What is RAG
Next: M7-C — RAG with NotebookLM
Module 8: M8-A — Document-Based AI Assistants