core
Mental model of modern LLMs
Explain what an LLM does, why generation varies, and why hallucination is a natural failure mode.
learn.smmeyer.devllm-engineering
menu
chapter index
Start in order, or use the rubrics to find a useful entry point. Time includes assignments, constraint variations, and repeated drills. Reading the notes is one part of that budget.
Course project: Harbor · build and evaluate a support assistant
Keep one project document and a folder of fictional input/output fixtures for Harbor, a workspace-scoped support assistant. Start with ticket triage and policy-backed drafts at 20 requests/minute, 1,500 input and 200 output tokens/request. Human users remain responsible for consequential actions until you design explicit permissions and approval. Later chapters add retrieval, controlled tools, durable runs, evaluation, security, and scale. Use paper, a text editor, and mocked results; paid APIs are optional. Preserve first attempts, evidence, and revisions within the existing practice budget.
Start the cumulative project →What do the depth labels mean?
Core builds essential vocabulary. Deep traces mechanisms. Studio applies them to a design. Patterns compares recurring structures. Capstone combines the course. Current covers update-sensitive platform contracts. Fluency practices concise explanations.
01Foundations
3 chapters · ~8h practice
01
~2h
4 checks
02
core
Transformer fundamentals
Explain attention and the transformer stack intuitively, including long-context cost.
~3h
4 checks
03
fluency
How models are trained
Describe pretraining, post-training, synthetic data, and evaluation without drifting into researcher-only detail.
~3h
4 checks
02Serving
1 chapters · ~4h practice
04
deep
Inference & serving
Trace an API request through prefill, decode, KV cache, batching, parallelism, and scheduling.
~4h
4 checks
03Application core
3 chapters · ~10h practice
05
core
Prompting as software engineering
Treat prompts as versioned specifications with schemas, tests, and explicit trust boundaries.
~3h
4 checks
06
core
Tool use & function calling
Build a safe tool lifecycle from schema exposure through validated execution and grounded continuation.
~3h
4 checks
07
deep
Agents
Design bounded model–action–observation loops with state, checkpoints, approvals, and failure recovery.
~4h
4 checks
04Knowledge systems
3 chapters · ~9h practice
08
core
Retrieval-augmented generation
Build and evaluate a permission-aware retrieval pipeline from ingestion through grounded output.
~4h
4 checks
09
core
Embeddings
Reason about embedding applications, distance metrics, indexing, and approximation tradeoffs.
~2h
4 checks
10
core
Context & memory
Allocate context and design safe memory that remains relevant, fresh, and user-controlled.
~3h
4 checks
05Production AI
3 chapters · ~11h practice
11
core
Evaluation
Build an eval-driven development loop across datasets, graders, human calibration, and online outcomes.
~4h
4 checks
12
core
Reliability & failure modes
Wrap probabilistic and upstream failures in deterministic recovery and safe degradation.
~3h
4 checks
13
deep
Safety & security
Threat-model model inputs, retrieved content, tools, data, and autonomy with defense in depth.
~4h
4 checks
06Product systems
2 chapters · ~6h practice
14
core
Building LLM products
Choose models and UX patterns across intelligence, latency, cost, streaming, feedback, and failure.
~3h
4 checks
15
patterns
LLM application architectures
Select the smallest architecture pattern that fits the task and autonomy level.
~3h
4 checks
07Agent systems
2 chapters · ~8h practice
16
deep
Coding agents & software engineering
Design a code-changing loop that understands repositories, edits safely, verifies, and recovers over long horizons.
~4h
4 checks
17
deep
LLM infrastructure at scale
Design gateways, queues, caches, quotas, tenant isolation, and capacity around scarce model compute.
~4h
4 checks
08LLM platforms
1 chapters · ~4h practice
18
current
Modern LLM platform concepts
Navigate the durable primitives exposed by modern LLM platforms and reason about provider evolution without memorizing a release snapshot.
~4h
4 checks
09Interview mode
3 chapters · ~13h practice
19
core
AI-native engineering judgment
Place probabilistic behavior only where it creates value, then measure and bound it.
~3h
4 checks
20
studio
AI-native design exercises
Practice full AI-native designs across storage, inference, agents, retrieval, memory, evals, routing, voice, and abuse.
~6h
4 checks
21
capstone
Interview-level LLM fluency
Explain, critique, and design AI systems precisely at software-engineer depth.
~4h
4 checks