skip to content

learn.smmeyer.devllm-engineering21

menu

llm-engineering/21

interview mode · capstone · ~4h practice

Interview-level LLM fluency

outcome

Explain, critique, and design AI systems precisely at software-engineer depth.

Plan a practice session · ~4h including exercises
Read + trace the worked example
25 min
Assignment + constraint variations
135 min
Timed drill + repeated explanations
60 min
Closed-book recall + self-review
20 min

This is a suggested practice budget, including drawing, variations, and repeated drills. Reading is the first block. Split the work across sessions and use the rubric to judge when to move on.

why this matters

Your goal is not to impersonate an ML researcher. It is to show a grounded mental model, serving intuition, product judgment, and honest boundaries—then connect them to reliable software design.

coverage

  • Explain: LLMs, attention, inference bottlenecks.
  • Quantify: latency stages, token cost, context and throughput.
  • Compare: RAG, prompting, tuning, agents.
  • Diagnose: model vs retrieval vs orchestration vs tool failure.
  • Secure: permissions, sandboxing, approval.
  • Critique: find missing evals, state, fallbacks, and controls.
  • Boundary: know when training-side detail is outside SWE scope.

mental model

Clear explanation + system diagnosis + bounded design = interview fluency.

  1. 1 →Explain

    First principles

  2. 2 →Diagnose

    Layer + evidence

  3. 3 →Design

    Tradeoffs + controls

  4. 4 →Bound

    Honest unknowns

Read in numbered order: explain → diagnose → design → bound.

lessons

01Build the five-minute stack

Explain tokens and next-token distributions; attention as learned information mixing; prefill vs decode and KV memory; retrieval/tools as external grounding/action; evals and deterministic controls around variable output.

Build an explanation in four moves: mechanism, concrete example, engineering consequence, and limitation. For KV cache, explain reused intermediate keys and values, calculate one sequence's memory, connect it to concurrency limits, then state the layout assumptions. That structure is short enough to speak and deep enough to support a follow-up.

Choose depth for the audience. A backend engineer may want the effect on queues and memory; a product manager may need the consequence for waiting time and cost. Keep the mechanism accurate while changing vocabulary. Avoid substituting a slogan such as “attention finds important words” for an explanation of learned mixing over allowed positions.

  • Use one diagram and one product consequence
  • Avoid magical language
  • Answer the level asked before going deeper

02Diagnose by layer

A bad answer may come from missing model capability, weak prompt, poor retrieval, stale context, incorrect tool, orchestration bug, policy block, or UX mismatch. Ask for traces and counterfactual tests.

Start diagnosis with a reproducible case and a trace of versions, context, tool observations, and final state. Was the needed evidence absent, filtered out, truncated, ignored, or contradicted? Did the tool proposal fail validation, authorization, execution, or result interpretation? These distinctions suggest different experiments and prevent changing the model for every bad outcome.

Use counterfactual tests: supply gold evidence, replace a tool result with a known correct fixture, run a deterministic parser independently, or compare a different model under the same inputs. Interpret each result narrowly. One repaired example suggests a cause to investigate; it does not establish the production failure rate or prove the next release is safe.

  • Swap evidence/model/tool one at a time
  • Reproduce with stored versions
  • Distinguish availability from quality

03State your boundary well

Say: ‘I do not know the optimizer-level detail, but for the serving/product system I would reason from these constraints…’ Then proceed concretely. Honest scope plus strong adjacent reasoning beats bluffing.

State the precise unknown and a verification path. “I have not verified this provider's current tool support; I would check the official capability documentation and a contract probe” leaves a clear next step. Continue the design with explicit branches for supported and unsupported capability. Do not invent a limit merely to avoid a pause.

After an oral circuit, review where your explanation omitted a unit, confused context with weights, treated similarity as probability, or asserted success without evidence. Rewrite only the weak segment and record again without notes. A final explanation should help another engineer make a decision and know what to measure next, not merely recognize the vocabulary.

  • Know vocabulary without pretending research depth
  • Connect to distributed systems expertise
  • Ask what evidence would resolve uncertainty

worked example

Explain an unfamiliar detail without inventing it

An interviewer asks for a provider's exact current cache pricing and whether a specific model supports a tool mode. You know the durable architecture but have not verified those facts.

  1. State the known mechanism: reuse can reduce repeated prefill work, and tool support is a capability contract.
  2. Identify the unknown facts precisely and name the official documentation or a small contract probe needed to verify them.
  3. Continue the design using explicit alternatives: if the capability exists, use the adapter; otherwise choose a compatible model or change the workflow.
  4. Give the product consequence in one sentence and avoid filling the gap with an invented number or guarantee.

Design decision: Demonstrate reliable reasoning, clear assumptions, and a concrete verification path even when a current detail is unknown.

Limit: Honest uncertainty still needs forward progress. It should lead to a testable assumption or alternative rather than ending the design conversation.

Teaching scenario · traffic, prices, and capacity values are illustrative.

assignment

Produce: Two recordings of a six-topic oral circuit and a revision sheet with timestamps.

Paper, a text editor, or a spreadsheet is enough. Model access and paid services are optional.

  1. Explain LLM generation, attention, inference, RAG, safe tool use, and unknown-detail recovery in two minutes each.
  2. For every explanation, name a mechanism, a failure mode, a product consequence, and a limitation; use the drill timers.
  3. Review the first circuit for jargon, unsupported absolutes, and missing units, rehearse weak segments, then record a second circuit without reading notes.

Change a constraint

  • Compress each explanation to one minute.
  • Answer the same question for a backend engineer and a product manager.

Self-review rubric

  • The mechanism is technically accurate and understandable.
  • The explanation connects to a design decision.
  • Unknown current facts have an explicit verification path.

If a criterion is missing, revise that part of the artifact before rating your practice below.

Compare a sample artifact after your attempt

One defensible answer under these assumptions. Compare the reasoning and checks with your work; other designs can satisfy the same contract.

The circuit tests explanations and honest recovery of unknown details. Timings below are fictional examples of useful self-review notes.

Six topics: generation / attention / serving / RAG / tool safety / unknown detail
Each: mechanism → example → consequence → limitation, within 2 minutes
First recording notes:
01:10 calls probability truth → replace with conditional token likelihood
04:20 omits KV heads → correct formula + 1 GiB example
07:00 says citations guarantee truth → distinguish support/current authority
09:30 unknown price → state unknown, official docs/probe, design alternatives
Second recording: explain without notes; cite repaired timestamps
Transfer: compress to 1 minute, then explain to backend and product audiences

The listener should understand a mechanism well enough to make a decision. Use verified durable facts and label illustrative numbers. For unknown current support or pricing, state the precise question and verification path, then continue with conditional designs rather than inventing a number. A score improves only when the new recording shows the repaired reasoning.

Check your artifact

  • Every explanation includes a concrete consequence and limitation.
  • No explanation confuses context with weights, similarity with probability, or proposed effects with receipts.
  • The second circuit corrects observed weaknesses and works for a changed audience.

A tempting mistake: Vocabulary fluency can hide a missing causal explanation. Ask what changes when one input or constraint changes.

cumulative practice

Harbor · build and evaluate a support assistant

Project brief

Keep one project document and a folder of fictional input/output fixtures for Harbor, a workspace-scoped support assistant. Start with ticket triage and policy-backed drafts at 20 requests/minute, 1,500 input and 200 output tokens/request. Human users remain responsible for consequential actions until you design explicit permissions and approval. Later chapters add retrieval, controlled tools, durable runs, evaluation, security, and scale. Use paper, a text editor, and mocked results; paid APIs are optional. Preserve first attempts, evidence, and revisions within the existing practice budget.

Keep one project document across this track. Reconstruct earlier reasoning before opening notes, then revise the same artifact. Use part of the assignment and review budget for this work. If you enter mid-course, sketch the earlier decisions first using the linked chapters.

Add to your project: Harbor handoff: a concise design explanation, evidence pack, and targeted review plan.

Explain the finished project in five minutes without notes, then answer one follow-up on generation, serving, retrieval, controlled actions, and evaluation. State an unknown provider detail with a concrete verification path. Identify a weakness from the capstone and choose a specific future retrieval exercise.

Compare cumulative reasoning after your attempt

Explain why Harbor supplies authorized evidence, validates structured proposals, separates durable effects from prose, budgets token work, and gates changes with task-level cases. Support the explanation with your packet, fixtures, failure traces, and review evidence. Unknown provider details lead to a dated capability check or probe and an explicit alternative. A useful next exercise targets one weak mechanism under a changed constraint, rather than rereading every chapter.

Keep your first attempt and revision together. If your answer differs, identify the assumption or mechanism that explains the difference before rating your recall.

senior signal

Existing software-engineering experience is an asset: translate service ownership, client/server contracts, release mechanisms, and production operations into examples of boundaries, rollouts, observability, and deterministic control around model behavior.

failure patterns

  • Buzzword chains without causal explanation
  • Bluffing training details
  • Describing agents without state, tools, or stop conditions
  • Treating a model failure as the only possible cause

knowledge checks

Start with these quick checks, then explain the closed-book questions in your own words.

  1. 01Best response to unknown training detail?

    select an option to check your answer

  2. 02A bad RAG answer should first be localized across…

    select an option to check your answer

closed-book recall

Close the worked example and explain these from memory. Answers are self-reviewed. Drafts stay on this page; save anything you want to keep in chapter notes.

Rate the assignment and recall together

Use the assignment rubric as evidence. Completion is your own judgment; this score schedules practice.

interview drill

Record a 12-minute oral exam: LLM in 2 min, attention in 2, inference in 2, RAG tradeoffs in 2, safe agent in 2, unknown-depth recovery in 2.

  • 12 min
  • no notes
  • concrete examples
  • one honest boundary
compare a strong approach after your attempt

Use the mental-model diagrams from chapters 1, 2, 4, 8, and 13. For each, define mechanism, consequence, and failure. End with an unknown training-detail question and practice pivoting to how you would measure or design around it.

Compare decisions and failure behavior, then repeat with a changed constraint. Keep your own wording.

Use the timed mock workspace →

references

Sources offer deeper context. Provider capabilities and API contracts can change; check current official docs before implementing.

notes

saved locally