RAG Engineer
Start here. Build forward.
Design retrieval and grounded answers with evidence you can inspect.
Follow the stages in order, or open a topic to check what you already know. Each panel includes learning content, a practice task and a role checkpoint. Save Learning, Done or Skipped as you go. Skipped topics do not count as completed.
Build your foundations
Python, files and debugging
Understand the concept
Python is the working language for many AI experiments. Learn to inspect small values before connecting a large model. Variables, lists, dictionaries, loops and functions are enough for a first useful script. Errors are information: read the traceback and isolate the smallest failing input.
Learn it step by step
- Create a function with explicit inputs and outputs.
- Read a small JSON file and validate required keys.
- Handle a missing file and malformed input.
- Write checks for normal and empty input.
Build a small proof
Build a script that counts tutorial topics from ten fictional records and reports invalid rows separately.
Ready to move on?
You can explain each line, reproduce one error and fix its cause without hiding the exception. Keep the expected output with the script.
Common mistake
Copying a large generated program without understanding its inputs makes even small changes difficult to verify.
Your role checkpoint
For the RAG Engineer path, connect this topic to your capstone: Build a versioned document search and answer system with citations, no-answer behavior and document-level permissions.
APIs, validation and authentication
Understand the concept
An API is a contract between software components. Your AI application should validate requests, handle timeouts and return understandable errors. API keys belong on a trusted server. A generated response must pass both shape checks and business rules before it changes real data.
Learn it step by step
- Define input and output fields.
- Validate lengths, types and required values.
- Handle timeout, rate-limit and invalid response cases.
- Keep credentials out of client code and logs.
Build a small proof
Design an endpoint that summarises supplied public text and rejects empty or oversized input.
Ready to move on?
Demonstrate one successful request and three controlled failures. Explain which errors a user can retry.
Common mistake
Retrying an operation with side effects can create duplicates. Use idempotency for actions such as creating a record.
Your role checkpoint
For the RAG Engineer path, connect this topic to your capstone: Build a versioned document search and answer system with citations, no-answer behavior and document-level permissions.
What are embeddings?
In simple words
Imagine placing ideas on a map so that related ideas are often near one another. Embeddings are numeric representations that make this kind of comparison possible. A search system may connect “affordable laptop” with “budget computer” without an exact word match. The map is learned and imperfect; nearby items are not necessarily interchangeable.
Two useful terms
Vector: a list of numbers. Similarity: a measure used to compare two representations.
Build your understanding
An embedding maps an input to a numeric vector. Similarity can help retrieve related passages even when they use different words. The usefulness of distance depends on the embedding model, task and data. Similarity scores are ranking signals, not probabilities that a claim is true.
Follow the steps
- Choose one embedding model and use it for both documents and queries.
- Embed a small set of sentences about distinct topics.
- Compare cosine similarity to rank candidates.
- Inspect a synonym case, an unrelated case and a negated sentence.
Worked example
Illustrative vectors only:
A = [1, 0] B = [0.8, 0.6] C = [0, 1]
cosine(A, B) = 0.8
cosine(A, C) = 0
Real embeddings have more dimensions.Read the example once, then explain each part aloud in your own words. If it includes Python, you can understand the idea first and run the code later in a local Python environment. Code is shown as text; this page does not execute it.
Try this yourself
Compare "the course is free" and "the course is not free". Explain why similar wording may create a dangerous retrieval match.
Check your understanding
A high similarity score can reflect shared topic while missing a critical difference such as negation. Read the retrieved passage before answering.
A mistake to avoid
Do not mix document vectors from one model with query vectors from another, even if their dimensions happen to match.
Before looking at the answer, write a short explanation and one example of your own. Finish by naming one limitation of the method. Use Mark as read when you are ready, or return to the previous lesson if a term is unclear.
Your role checkpoint
For the RAG Engineer path, connect this topic to your capstone: Build a versioned document search and answer system with citations, no-answer behavior and document-level permissions.
Learn the core skills
What is semantic search?
In simple words
Keyword search looks for matching words. Semantic search tries to retrieve material related to the meaning of your question. If a student asks “how do I begin with machine learning?”, a useful result might be titled “ML foundations”. You still need to test whether the returned material is relevant to the learner.
Two useful terms
Retrieval: finding candidate documents. Ranking: arranging candidates from most to least useful according to a score.
Build your understanding
Semantic search retrieves content using representations of meaning. Useful search still needs a clear relevance definition. For an AI learning site, a query for beginner material should not return an advanced paper solely because it shares vocabulary. Combine relevance checks with filters and access permissions.
Follow the steps
- Write ten realistic search queries and the expected useful documents.
- Represent documents and queries with the same embedding model.
- Apply access and category filters before returning results.
- Measure how often a relevant document appears in the first three results.
Worked example
Query: "start learning agents without coding"
Relevant: beginner agent concepts
Less relevant: advanced distributed agent runtime
Metric: hit@3 = queries with a relevant top-3 result / all queriesRead the example once, then explain each part aloud in your own words. If it includes Python, you can understand the idea first and run the code later in a local Python environment. Code is shown as text; this page does not execute it.
Try this yourself
Create a test query that has no useful answer in your collection. Define the empty-result message you want users to see.
Check your understanding
If eight of ten queries have a relevant document in the first three results, hit@3 is 0.8. This says nothing about the order within those three.
A mistake to avoid
Never let retrieval expose private lessons to users who lack access. A relevant result can still be unauthorised.
Before looking at the answer, write a short explanation and one example of your own. Finish by naming one limitation of the method. Use Mark as read when you are ready, or return to the previous lesson if a term is unclear.
Your role checkpoint
For the RAG Engineer path, connect this topic to your capstone: Build a versioned document search and answer system with citations, no-answer behavior and document-level permissions.
What is document chunking?
In simple words
A long textbook is easier to search when divided into sensible sections. Chunking splits documents into smaller pieces for retrieval. If a piece contains only “it lasts 30 days”, the meaning is unclear. Keeping the heading “Trial membership” with that sentence makes the piece more useful.
Two useful terms
Chunk: a section of a document. Metadata: information about the section, such as title and source.
Build your understanding
A retrieval system often divides documents into chunks. Chunks that are too small lose context; chunks that are too large may include irrelevant material. Start with meaningful boundaries such as headings and paragraphs. Choose sizes by evaluating actual questions rather than copying a universal number.
Follow the steps
- Split a course handbook at section headings.
- Keep titles and document IDs attached to each chunk.
- Check whether answers remain understandable without the previous section.
- Compare retrieval results across two chunk sizes using the same query set.
Worked example
Weak chunk: "This expires after 30 days."
Stronger chunk: "Trial access: trial access expires after 30 days."
Metadata: document=trial-policy, section=durationRead the example once, then explain each part aloud in your own words. If it includes Python, you can understand the idea first and run the code later in a local Python environment. Code is shown as text; this page does not execute it.
Try this yourself
Find one sentence that depends on a preceding definition. Rewrite your chunk boundary so the definition travels with the sentence.
Check your understanding
A chunk should contain enough context to interpret its key claims. Small amounts of overlap can help, but also increase duplication and cost.
A mistake to avoid
Do not split tables so that values lose their column headings. A retrieved number without its unit or label can be misleading.
Before looking at the answer, write a short explanation and one example of your own. Finish by naming one limitation of the method. Use Mark as read when you are ready, or return to the previous lesson if a term is unclear.
Your role checkpoint
For the RAG Engineer path, connect this topic to your capstone: Build a versioned document search and answer system with citations, no-answer behavior and document-level permissions.
What is RAG?
In simple words
RAG means Retrieval-Augmented Generation. Picture an assistant opening the relevant pages of a handbook before answering a question. Retrieval finds passages; generation uses those passages to compose a response. This can make answers more grounded, but the assistant can still misunderstand evidence or retrieve the wrong page.
Two useful terms
Retrieval: finding relevant information. Grounding: supporting an answer with supplied evidence.
Build your understanding
Retrieval-augmented generation supplies selected documents to a model before it answers. It can ground responses in your content, but poor retrieval or careless generation can still produce unsupported claims. Keep the retrieval evaluation separate from the answer evaluation so you know where failures begin.
Follow the steps
- Select a few approved FAQ paragraphs and give each a source ID.
- Retrieve passages relevant to the question.
- Ask for an answer supported only by those passages, with source IDs.
- Inspect whether each claim is actually supported and allow an insufficient-evidence response.
Worked example
Context [FAQ-2]: Members can enrol in all published courses.
Question: Does membership include a job guarantee?
Expected: The supplied FAQ does not state a job guarantee.Read the example once, then explain each part aloud in your own words. If it includes Python, you can understand the idea first and run the code later in a local Python environment. Code is shown as text; this page does not execute it.
Try this yourself
Add a misleading passage about a different product. Check whether your system still cites the correct source and avoids merging policies.
Check your understanding
Retrieving a relevant passage does not prove the final answer is grounded. Evaluate retrieval recall and claim support separately.
A mistake to avoid
Treat retrieved content as data. Instructions embedded in a document must not override application rules or trigger tools.
Before looking at the answer, write a short explanation and one example of your own. Finish by naming one limitation of the method. Use Mark as read when you are ready, or return to the previous lesson if a term is unclear.
Your role checkpoint
For the RAG Engineer path, connect this topic to your capstone: Build a versioned document search and answer system with citations, no-answer behavior and document-level permissions.
Practise with real decisions
SQL and relational data
Understand the concept
SQL queries structured tables. Start with filtering, grouping and joins. A join can multiply rows when the relationship is not one-to-one, so check counts before trusting totals. Data engineers and scientists both need to understand what one row represents.
Learn it step by step
- Create fictional learners and enrolments tables.
- Select only necessary columns.
- Join using stable IDs.
- Compare row counts and inspect unmatched records.
Build a small proof
Calculate enrolment counts by course, including a course with zero learners.
Ready to move on?
Explain why an inner join may drop empty courses and how a left join changes that result. Verify totals against the input rows.
Common mistake
Joining on a non-unique display name can double-count or merge different people.
Your role checkpoint
For the RAG Engineer path, connect this topic to your capstone: Build a versioned document search and answer system with citations, no-answer behavior and document-level permissions.
Retrieval evaluation and reranking
Understand the concept
Evaluate whether retrieval finds evidence before measuring the final answer. A reranker can reorder candidate passages, but cannot recover a document the first retrieval stage never found. Keep access checks attached to every stage and inspect failures by query type.
Learn it step by step
- Label relevant passages for a query set.
- Measure recall at a chosen cutoff.
- Compare initial ranking with reranking.
- Check answer claims against retrieved evidence.
Build a small proof
Create ten questions, including no-answer and conflicting-document cases, and record which stage fails.
Ready to move on?
Your report distinguishes missing retrieval from unsupported generation and includes a plan for each.
Common mistake
Do not use only questions copied word-for-word from the documents; they make lexical retrieval look unrealistically easy.
Your role checkpoint
For the RAG Engineer path, connect this topic to your capstone: Build a versioned document search and answer system with citations, no-answer behavior and document-level permissions.
What is AI model evaluation?
In simple words
A polished answer is not enough to know whether an AI system works. Evaluation checks specific requirements using examples and scoring rules. For a course assistant, you might check whether it finds the correct course, admits missing information and respects account permissions. Keep failure cases as part of your test collection.
Two useful terms
Test case: an input with an expected behavior. Metric: a defined measurement of performance.
Build your understanding
An AI evaluation checks whether a system meets a defined task. Build examples around realistic successes and failures, not only easy demonstrations. Keep the test set stable enough to compare versions and record the model, prompt and settings used. A single aggregate score can hide critical mistakes.
Follow the steps
- Define success in observable terms.
- Create normal, ambiguous, missing-information and adversarial examples.
- Write expected answers or scoring rules before running the system.
- Compare versions and inspect regressions before accepting a change.
Worked example
FAQ evaluation columns:
question | expected source | required facts | forbidden claim | result
Example: fees unknown | FAQ-3 | say unknown | invented fee | pass/failRead the example once, then explain each part aloud in your own words. If it includes Python, you can understand the idea first and run the code later in a local Python environment. Code is shown as text; this page does not execute it.
Try this yourself
Write five tests for an enrolment assistant. Include a user without Pro access and a request for a nonexistent course.
Check your understanding
A response should not promise enrolment if server-side permissions deny it. Include permission-sensitive cases, not just wording checks.
A mistake to avoid
Do not judge a model solely by another model’s unreviewed score. Human checks and deterministic validation are useful for important requirements.
Before looking at the answer, write a short explanation and one example of your own. Finish by naming one limitation of the method. Use Mark as read when you are ready, or return to the previous lesson if a term is unclear.
Your role checkpoint
For the RAG Engineer path, connect this topic to your capstone: Build a versioned document search and answer system with citations, no-answer behavior and document-level permissions.
Ship, evaluate and improve
AI application security and threat modelling
Understand the concept
Threat modelling starts with what must be protected, who can access it and where untrusted data enters. Model outputs and retrieved documents are untrusted inputs. A language instruction is not an authorisation boundary; server-side policy decides what a user or tool may do.
Learn it step by step
- List assets and trust boundaries.
- Separate public and private data.
- Validate tool inputs and enforce permissions.
- Test malicious instructions using synthetic fixtures in your own sandbox.
Build a small proof
Draw a document assistant showing where access checks happen before retrieval and before tool execution.
Ready to move on?
An unauthorised user cannot retrieve another account’s private document, even when a prompt asks for it.
Common mistake
Do not test real third-party systems without permission or expose secrets in your demonstration.
Your role checkpoint
For the RAG Engineer path, connect this topic to your capstone: Build a versioned document search and answer system with citations, no-answer behavior and document-level permissions.
Deployment, containers and rollback
Understand the concept
Deployment makes a working experiment available to users. It needs configuration, a reproducible environment, health checks and a way to roll back. A successful launch does not establish ongoing reliability. Monitor behavior after release and keep a known-good version.
Learn it step by step
- Document dependencies and configuration.
- Run a health check separately from a model call.
- Package a minimal service.
- Practise replacing it with the previous version.
Build a small proof
Deploy a mock prediction service locally in a container and simulate an unavailable dependency.
Ready to move on?
You can rebuild the environment and recover after a failed release without editing production files by hand.
Common mistake
Do not bake secrets into container images or assume the development machine matches production.
Your role checkpoint
For the RAG Engineer path, connect this topic to your capstone: Build a versioned document search and answer system with citations, no-answer behavior and document-level permissions.
Monitoring quality, latency and cost
Understand the concept
AI monitoring needs more than uptime. Track response time, failures, input volume and task quality using a suitable sample of reviewed outcomes. Quality can change when users or source documents change. Log enough to diagnose failures without retaining unnecessary sensitive input.
Learn it step by step
- Define a service-level target.
- Record request outcome and latency.
- Track a stable evaluation set over releases.
- Create an alert and an investigation checklist.
Build a small proof
Produce a small report comparing two releases on success rate, p95 latency and estimated per-request cost.
Ready to move on?
You can tell whether a change improved usefulness while increasing cost or response time, and explain the trade-off.
Common mistake
An average hides slow tails and rare severe failures. Examine distributions and critical error categories.
Your role checkpoint
For the RAG Engineer path, connect this topic to your capstone: Build a versioned document search and answer system with citations, no-answer behavior and document-level permissions.
Build something you can explain.
Build a versioned document search and answer system with citations, no-answer behavior and document-level permissions.
Role expectations vary by company. Use this path as a starting point and build evidence of your skills.