The full RAG flow

You've seen RAG at a high level, you know how to chunk, you know how to generate embeddings. Time to stitch everything together into a single flow and walk through what actually happens from "user asks a question" to "Claude returns an answer based on the right piece of the document." To keep the math intuitive we'll use a deliberately simplified two-dimensional embedding model that you'd never ship in production, but which makes the geometry of semantic search impossible to miss.

Step 1 — Chunk the source text

Imagine our source document has exactly two sections:

Section 1 — Medical Research

"This year saw significant strides in our understanding of XDR-47, a 'bug' we have not seen before."

Section 2 — Software Engineering

"This division dedicated significant effort to studying various infection vectors in our distributed systems."

Two chunks. Notice the mischief: the medical section contains the word "bug" (the pathogen), and the software section contains "infection vectors" (the security concept). A keyword search for "bug" would return the medical section even though a user asking "how many bugs did engineers fix this year" wants the software section. This is exactly what semantic search is supposed to handle.

Step 2 — Generate embeddings

In a real system, an embedding is 1,024 dimensions of floats and not remotely inspectable. For this lesson, pretend we have a magic two-dimensional embedding model with two known axes:

• Dimension 1: how much the text is about the medical field (0 to 1)
• Dimension 2: how much the text is about software engineering (0 to 1)

Feed the chunks through the magic model:

• Medical section → [0.97, 0.34] — heavily medical, small software signal because of "bug."
• Software section → [0.30, 0.97] — heavily software, small medical signal because of "infection vectors."

In a real pipeline you'd call generate_embedding(chunk_text, input_type="document") for each chunk. The values would be longer and not human-readable, but the principle is identical.

Normalisation

Embedding APIs typically return normalised vectors — vectors scaled so that their magnitude is exactly 1.0. The math isn't important for now; assume the API does this for you and your vectors always sit on the unit circle (in 2D) or unit sphere (in higher dimensions).

After normalisation:

• Medical → [0.944, 0.331]
• Software → [0.295, 0.955]

Both vectors now have length 1, which makes the similarity math in the next step clean.

Step 3 — Store in a vector database

A vector database is a specialised data store optimised for exactly one thing: holding a large collection of embeddings and efficiently answering the question "which stored embedding is closest to this query embedding?" Popular options include pgvector (Postgres extension), Qdrant, Pinecone, Weaviate, and Chroma. Each has its own API, but they all accept embeddings as input and return similarity-ranked matches as output.

You store each chunk along with its embedding. The preprocessing phase ends here. Everything so far has happened ahead of time, before any user ever types a question. The chunks are chunked, the embeddings are embedded, the database is populated. Now we wait.

Step 4 — Process the user query

A user asks:

"I'm curious about the company. In particular, what did the software engineering department do this year?"

Run the query through the same embedding model — critical that it's the same model, so the query lives in the same vector space as the stored chunks.

The query embedding might look like [0.1, 0.89] — low medical, high software engineering. After normalisation: [0.112, 0.993].

Step 5 — Find similar embeddings

Hand the query embedding to the vector database and ask for the nearest stored chunks. The database uses cosine similarity to score each candidate.

Cosine similarity, briefly

Cosine similarity measures the cosine of the angle between two vectors. The insight: two vectors pointing in similar directions have a small angle between them (and a cosine close to 1); two vectors pointing in very different directions have a large angle (cosine close to 0 or even -1).

• Cosine = 1 — vectors point in exactly the same direction. High similarity.
• Cosine = 0 — vectors are perpendicular. No relationship.
• Cosine = -1 — vectors point in opposite directions. Maximally different.

For our example:

• Query [0.112, 0.993] vs Software [0.295, 0.955] → cosine similarity ≈ 0.983. Nearly identical direction. Very close match.
• Query [0.112, 0.993] vs Medical [0.944, 0.331] → cosine similarity ≈ 0.398. Much less similar.

The database returns the software section as the top match. Exactly what the user wanted, even though the medical section contains the word "bug" that confused keyword search.

Cosine distance

You'll see "cosine distance" in most vector database documentation. It's defined as 1 - cosine_similarity, which inverts the scale:

• Cosine distance close to 0 — high similarity
• Cosine distance close to 1 (or higher) — low similarity

This is easier to reason about for many people because "closer means smaller." When you're configuring a vector database, double-check which metric it uses — cosine similarity and cosine distance are interchangeable but you want to know which direction "better" is in.

Step 6 — Construct the final prompt

Now that you have the most relevant chunk, compose the final Claude prompt:

Answer the user's question about the financial document.

<user_question>
How many bugs did engineers fix this year?
</user_question>

<report>
## Section 2: Software Engineering
This division dedicated significant effort to studying various
infection vectors in our distributed systems.
</report>

The whole document didn't make it into the prompt. Just the relevant chunk did, wrapped in XML tags for clean boundaries (module 5's XML structuring pays off here).

Send this prompt to Claude. Claude now has exactly the context it needs to answer correctly. The user gets a response grounded in the right paragraph. The entire pipeline ran without ever putting the medical research content into Claude's context at all.

Putting it all together

The full flow end to end:

1. Chunk the document during preprocessing.
2. Embed each chunk with a query-agnostic model.
3. Store the chunks and their embeddings in a vector database.
4. Embed the user's query with the same model.
5. Search the database for the chunks with highest cosine similarity to the query.
6. Construct a prompt with the question plus the top-ranked chunks.
7. Call Claude with that prompt and return the answer.

In the next lesson you'll implement the whole thing in code against a real document — no more toy two-dimensional embeddings, real retrieval against real chunks.