LLM Engineer
Start here. Build forward.
Understand language models, adaptation and the constraints of serving them.
Follow the stages in order, or open a topic to check what you already know. Each panel includes learning content, a practice task and a role checkpoint. Save Learning, Done or Skipped as you go. Skipped topics do not count as completed.
Build your foundations
Python, files and debugging
Understand the concept
Python is the working language for many AI experiments. Learn to inspect small values before connecting a large model. Variables, lists, dictionaries, loops and functions are enough for a first useful script. Errors are information: read the traceback and isolate the smallest failing input.
Learn it step by step
- Create a function with explicit inputs and outputs.
- Read a small JSON file and validate required keys.
- Handle a missing file and malformed input.
- Write checks for normal and empty input.
Build a small proof
Build a script that counts tutorial topics from ten fictional records and reports invalid rows separately.
Ready to move on?
You can explain each line, reproduce one error and fix its cause without hiding the exception. Keep the expected output with the script.
Common mistake
Copying a large generated program without understanding its inputs makes even small changes difficult to verify.
Your role checkpoint
For the LLM Engineer path, connect this topic to your capstone: Compare a prompting baseline with a justified adaptation or serving change, reporting quality, latency and limitations.
Linear algebra and optimisation intuition
Understand the concept
Vectors, matrices, derivatives and gradients provide the language for model calculations. Begin with small numeric examples and visual intuition. A gradient describes a local direction of change; optimisation uses it to adjust parameters against a loss. It does not guarantee a globally best solution.
Learn it step by step
- Multiply a small matrix and vector by hand.
- Plot a simple quadratic loss.
- Follow several gradient steps.
- Compare a learning rate that converges with one that overshoots.
Build a small proof
Use a spreadsheet or short Python script to minimise a one-variable quadratic and log each step.
Ready to move on?
Explain what the gradient, learning rate and loss each represent in your example.
Common mistake
Memorising symbols without relating them to a calculation makes debugging training much harder.
Your role checkpoint
For the LLM Engineer path, connect this topic to your capstone: Compare a prompting baseline with a justified adaptation or serving change, reporting quality, latency and limitations.
What is a large language model?
In simple words
A large language model, or LLM, is a model trained on language-related data at scale to predict or generate sequences of tokens. Chat assistants often use LLMs together with instructions, tools and other software. Fluent language does not guarantee correct facts or access to current information.
Words to know
LLM: large language model. Inference: using a trained model to produce an output.
Pause here and explain the definition in one sentence without copying it. The aim is to describe what the system does, not to memorise an impressive phrase.
A familiar example
Ask an assistant to rewrite a paragraph more simply. The model can produce a useful rewrite without consulting a search engine. Asking for today’s train timetable is different: that needs current reliable data, not only plausible text.
Break it down
- Identify the input: what information is available?
- Describe the operation or pattern being used.
- Identify the output and how you would check it.
- Name one mistake the system could make.
Change one detail in this example and consider whether the same idea still applies. This helps separate the underlying concept from one specific application.
Your turn
Write one task that uses information entirely inside your prompt and one that needs an external source.
Answer check
Summarising supplied notes can rely on the notes. Reporting a live train delay needs a current source. Whether an assistant can fetch that source depends on its available tools.
Explain it to a friend
Give a second example from college, a hobby or a fictional workplace. State what information goes in, what comes out and one limitation. If you can explain those three things in plain language, you are ready to move on. Mark this lesson as read to save your progress when signed in.
Your role checkpoint
For the LLM Engineer path, connect this topic to your capstone: Compare a prompting baseline with a justified adaptation or serving change, reporting quality, latency and limitations.
Learn the core skills
What is tokenisation?
In simple words
A language model does not simply receive text as one giant sentence. A tokenizer splits it into units called tokens. Think of dividing a chocolate bar into pieces before counting it, except token pieces do not have a fixed size. A short word might be one piece; another word might use several.
Two useful terms
Token: one unit processed by a model. Tokenizer: the method that splits text into those units.
Build your understanding
Tokenisation converts text into smaller units a model can process. Depending on the tokenizer, a unit may be a word, part of a word, punctuation or bytes. Token counts are model-specific. A whitespace split is an educational baseline, not an accurate substitute for a language model tokenizer.
Follow the steps
- Write examples containing English, Hindi, emoji and source code.
- Compare a simple whitespace split with the tokenizer for your chosen model.
- Record which strings create surprisingly many tokens.
- Estimate your input budget using that tokenizer before sending large documents.
Worked example
"AI is useful".split() -> ["AI", "is", "useful"]
This demonstrates whitespace splitting only.
A model tokenizer may split "usefulness" into several pieces.Read the example once, then explain each part aloud in your own words. If it includes Python, you can understand the idea first and run the code later in a local Python environment. Code is shown as text; this page does not execute it.
Try this yourself
Compare "hello", "Hello!" and "नमस्ते" with the same tokenizer. Explain why character length alone does not determine token count.
Check your understanding
The same text can produce different token counts for different tokenizers. You must specify the tokenizer when reporting a count.
A mistake to avoid
Do not assume one token equals one word, especially for multilingual text, URLs and code.
Before looking at the answer, write a short explanation and one example of your own. Finish by naming one limitation of the method. Use Mark as read when you are ready, or return to the previous lesson if a term is unclear.
Your role checkpoint
For the LLM Engineer path, connect this topic to your capstone: Compare a prompting baseline with a justified adaptation or serving change, reporting quality, latency and limitations.
What is a transformer in AI?
In simple words
A transformer is a model architecture that uses attention along with other components. Attention helps combine information from different positions in a sequence. In “river bank” and “bank account”, the surrounding words give different clues. This is an intuition for context, not a complete explanation of how a trained model works.
Two useful terms
Architecture: the organisation of a model. Attention: learned computation that weights information from different positions.
Build your understanding
Attention lets a model combine information from different positions using learned relevance weights. Transformers use attention with other components to process sequences. The representation of a word can change with context. This tutorial is an intuition exercise, not a full implementation of training or inference.
Follow the steps
- Write two sentences where the same word has different meanings.
- Identify nearby words that disambiguate it.
- Imagine assigning different relevance weights to those words.
- Explain how a context-sensitive representation differs from a fixed dictionary lookup.
Worked example
"She sat by the river bank."
"She deposited money at the bank."
The tokens "river" and "money" provide different context.
Attention is learned numeric computation, not a human reading rule.Read the example once, then explain each part aloud in your own words. If it includes Python, you can understand the idea first and run the code later in a local Python environment. Code is shown as text; this page does not execute it.
Try this yourself
Use "bat" in a sports sentence and an animal sentence. Identify the evidence that distinguishes the meanings.
Check your understanding
Context can change the useful representation of the same token. Attention weights alone are not a reliable explanation of a model’s complete decision.
A mistake to avoid
Do not interpret a fluent generated continuation as verified knowledge. Sequence modelling and fact-checking are different tasks.
Before looking at the answer, write a short explanation and one example of your own. Finish by naming one limitation of the method. Use Mark as read when you are ready, or return to the previous lesson if a term is unclear.
Your role checkpoint
For the LLM Engineer path, connect this topic to your capstone: Compare a prompting baseline with a justified adaptation or serving change, reporting quality, latency and limitations.
What is a neural network?
In simple words
A neural network combines numbers through connected layers to produce an output. Its weights are adjusted during training. Start with one small calculation before thinking about enormous models. The term “neural” is inspired by biology, but it does not establish that the system thinks or feels like a human brain.
Two useful terms
Weight: an adjustable multiplier. Activation: a function applied to a layer’s values.
Build your understanding
A neural network combines weighted inputs through layers and nonlinear transformations. Training adjusts weights to reduce a chosen loss. You can understand the basic computation with one neuron before learning larger architectures. More layers do not remove the need for careful data and evaluation.
Follow the steps
- Choose two inputs, two weights and a bias.
- Compute the weighted sum.
- Apply a simple activation such as ReLU.
- Compare the output with a target and describe the resulting error.
Worked example
x = [2, 3], weights = [0.5, -1], bias = 1
z = 2*0.5 + 3*(-1) + 1 = -1
ReLU(z) = max(0, z) = 0
This is a forward pass, not a complete training algorithm.Read the example once, then explain each part aloud in your own words. If it includes Python, you can understand the idea first and run the code later in a local Python environment. Code is shown as text; this page does not execute it.
Try this yourself
Change the bias to 3 and recompute the output. Explain which part of the computation changed.
Check your understanding
The new weighted sum is 1, so ReLU outputs 1. A bias shifts the activation boundary without changing the input weights.
A mistake to avoid
An analogy to a biological brain is only an analogy. It does not explain reliability, consciousness or general intelligence.
Before looking at the answer, write a short explanation and one example of your own. Finish by naming one limitation of the method. Use Mark as read when you are ready, or return to the previous lesson if a term is unclear.
Your role checkpoint
For the LLM Engineer path, connect this topic to your capstone: Compare a prompting baseline with a justified adaptation or serving change, reporting quality, latency and limitations.
Practise with real decisions
Fine-tuning and adaptation
Understand the concept
Fine-tuning changes a model’s parameters using task-specific training examples. It is different from supplying documents at inference time. First establish whether prompting or retrieval solves the problem. A training dataset needs permission, consistent labels and an evaluation split that reflects intended use.
Learn it step by step
- Define a baseline without tuning.
- Prepare a small, reviewed example set.
- Separate training and held-out evaluation.
- Compare usefulness, regressions and cost.
Build a small proof
Write an experiment plan for consistent support-response formatting using synthetic examples.
Ready to move on?
Your plan has a reason tuning might help and a test that could show it did not. Include cases outside the narrow training pattern.
Common mistake
Fine-tuning is not a dependable way to keep rapidly changing facts current.
Your role checkpoint
For the LLM Engineer path, connect this topic to your capstone: Compare a prompting baseline with a justified adaptation or serving change, reporting quality, latency and limitations.
Inference efficiency and model trade-offs
Understand the concept
Inference is running a trained model on new input. Latency, memory and throughput depend on model size, context, hardware and serving choices. Compare configurations on the same workload. A faster system is only useful if it still meets quality requirements.
Learn it step by step
- Choose a representative request set.
- Measure warm and cold behavior.
- Compare context lengths and batch sizes.
- Record quality alongside latency and memory.
Build a small proof
Make a small experiment table for a mock or locally available model; do not claim hardware results you did not measure.
Ready to move on?
You can explain a trade-off between response time, resource use and quality based on observed measurements.
Common mistake
A benchmark using tiny prompts may not represent long real user requests.
Your role checkpoint
For the LLM Engineer path, connect this topic to your capstone: Compare a prompting baseline with a justified adaptation or serving change, reporting quality, latency and limitations.
What is AI model evaluation?
In simple words
A polished answer is not enough to know whether an AI system works. Evaluation checks specific requirements using examples and scoring rules. For a course assistant, you might check whether it finds the correct course, admits missing information and respects account permissions. Keep failure cases as part of your test collection.
Two useful terms
Test case: an input with an expected behavior. Metric: a defined measurement of performance.
Build your understanding
An AI evaluation checks whether a system meets a defined task. Build examples around realistic successes and failures, not only easy demonstrations. Keep the test set stable enough to compare versions and record the model, prompt and settings used. A single aggregate score can hide critical mistakes.
Follow the steps
- Define success in observable terms.
- Create normal, ambiguous, missing-information and adversarial examples.
- Write expected answers or scoring rules before running the system.
- Compare versions and inspect regressions before accepting a change.
Worked example
FAQ evaluation columns:
question | expected source | required facts | forbidden claim | result
Example: fees unknown | FAQ-3 | say unknown | invented fee | pass/failRead the example once, then explain each part aloud in your own words. If it includes Python, you can understand the idea first and run the code later in a local Python environment. Code is shown as text; this page does not execute it.
Try this yourself
Write five tests for an enrolment assistant. Include a user without Pro access and a request for a nonexistent course.
Check your understanding
A response should not promise enrolment if server-side permissions deny it. Include permission-sensitive cases, not just wording checks.
A mistake to avoid
Do not judge a model solely by another model’s unreviewed score. Human checks and deterministic validation are useful for important requirements.
Before looking at the answer, write a short explanation and one example of your own. Finish by naming one limitation of the method. Use Mark as read when you are ready, or return to the previous lesson if a term is unclear.
Your role checkpoint
For the LLM Engineer path, connect this topic to your capstone: Compare a prompting baseline with a justified adaptation or serving change, reporting quality, latency and limitations.
Ship, evaluate and improve
AI application security and threat modelling
Understand the concept
Threat modelling starts with what must be protected, who can access it and where untrusted data enters. Model outputs and retrieved documents are untrusted inputs. A language instruction is not an authorisation boundary; server-side policy decides what a user or tool may do.
Learn it step by step
- List assets and trust boundaries.
- Separate public and private data.
- Validate tool inputs and enforce permissions.
- Test malicious instructions using synthetic fixtures in your own sandbox.
Build a small proof
Draw a document assistant showing where access checks happen before retrieval and before tool execution.
Ready to move on?
An unauthorised user cannot retrieve another account’s private document, even when a prompt asks for it.
Common mistake
Do not test real third-party systems without permission or expose secrets in your demonstration.
Your role checkpoint
For the LLM Engineer path, connect this topic to your capstone: Compare a prompting baseline with a justified adaptation or serving change, reporting quality, latency and limitations.
Deployment, containers and rollback
Understand the concept
Deployment makes a working experiment available to users. It needs configuration, a reproducible environment, health checks and a way to roll back. A successful launch does not establish ongoing reliability. Monitor behavior after release and keep a known-good version.
Learn it step by step
- Document dependencies and configuration.
- Run a health check separately from a model call.
- Package a minimal service.
- Practise replacing it with the previous version.
Build a small proof
Deploy a mock prediction service locally in a container and simulate an unavailable dependency.
Ready to move on?
You can rebuild the environment and recover after a failed release without editing production files by hand.
Common mistake
Do not bake secrets into container images or assume the development machine matches production.
Your role checkpoint
For the LLM Engineer path, connect this topic to your capstone: Compare a prompting baseline with a justified adaptation or serving change, reporting quality, latency and limitations.
Monitoring quality, latency and cost
Understand the concept
AI monitoring needs more than uptime. Track response time, failures, input volume and task quality using a suitable sample of reviewed outcomes. Quality can change when users or source documents change. Log enough to diagnose failures without retaining unnecessary sensitive input.
Learn it step by step
- Define a service-level target.
- Record request outcome and latency.
- Track a stable evaluation set over releases.
- Create an alert and an investigation checklist.
Build a small proof
Produce a small report comparing two releases on success rate, p95 latency and estimated per-request cost.
Ready to move on?
You can tell whether a change improved usefulness while increasing cost or response time, and explain the trade-off.
Common mistake
An average hides slow tails and rare severe failures. Examine distributions and critical error categories.
Your role checkpoint
For the LLM Engineer path, connect this topic to your capstone: Compare a prompting baseline with a justified adaptation or serving change, reporting quality, latency and limitations.
Build something you can explain.
Compare a prompting baseline with a justified adaptation or serving change, reporting quality, latency and limitations.
Role expectations vary by company. Use this path as a starting point and build evidence of your skills.