skip to content

learn.smmeyer.devllm-engineering07

menu

llm-engineering/07

application core · deep · ~4h practice

Agents

outcome

Design bounded model–action–observation loops with state, checkpoints, approvals, and failure recovery.

Plan a practice session · ~4h including exercises
Read + trace the worked example
30 min
Assignment + constraint variations
140 min
Timed drill + repeated explanations
50 min
Closed-book recall + self-review
20 min

This is a suggested practice budget, including drawing, variations, and repeated drills. Reading is the first block. Split the work across sessions and use the rubric to judge when to move on.

why this matters

An agent is useful when the path cannot be completely specified in advance but progress can be observed and actions can be bounded. It is not a synonym for any LLM call.

coverage

  • Loop: model → action → observation → replan.
  • State: working memory, durable run state, artifacts.
  • Horizon: long tasks, checkpoints, resumability.
  • Control: human approval, permissions, budgets.
  • Failure: retries, recovery, loop detection, dead ends.
  • Architecture: single/multi-agent, delegation, deterministic workflows.

mental model

A robust agent loop advances durable state under budgets and policy.

  1. 1 →Observe

    State + goal

  2. 2 →Plan

    Next bounded step

  3. 3 →Act

    Tool under policy

  4. 4 →Check

    Progress + stop

Read in numbered order: observe → plan → act → check.

lessons

01Use agents for uncertain paths

If a fixed DAG can express the workflow, use it: it is cheaper, easier to test, and more predictable. Use an agent when new observations change the next action and semantic judgment is central.

The useful agent distinction is control flow. In a workflow, the application knows the sequence of steps; a model may classify or extract inside one step. In an agent loop, the model chooses a next action from observations within a bounded goal. A fixed refund policy often belongs in a workflow. Investigating an unfamiliar repository may require adaptive search and inspection.

Compare both designs before choosing. A workflow has explicit transitions that are easier to test and recover. An adaptive loop can explore uncertain paths but introduces variable step count, cost, and failure. Give it the smallest tool set and task scope that permits useful exploration. A hybrid can keep a deterministic outer workflow while delegating one uncertain subproblem to a bounded loop.

  • Keep deterministic outer orchestration
  • Define completion evidence
  • Give the model the smallest useful action space

02State outlives context

Persist task state, tool results, artifacts, approvals, and checkpoints outside the prompt. Summarize or retrieve relevant history into context. A worker crash should not erase the run.

Persist the state needed to recover: run ID, original input, goal, policy versions, completed steps, tool attempts, approvals, and authoritative effects. The conversation is working context, not an adequate transaction log. A summarized conversation can omit the very detail needed to tell whether a side effect already happened.

Place checkpoints around effect boundaries. A worker may stop after a provider committed but before a local completion record was written. On restart, reconcile the operation using its stable identity; do not simply replay the last model instruction. Distinguish an unfinished read, an uncertain write, and a completed effect whose explanation was never delivered. Each resumes differently.

  • Use explicit state machines for lifecycle
  • Commit checkpoints after durable effects
  • Separate user memory from task scratch state

03Every loop needs a budget and stop rule

Limit wall time, tokens, cost, calls, repeated actions, and privileges. Detect non-progress, oscillation, and duplicate calls. Escalate to a human or return a partial result with evidence.

Bound steps, wall-clock time, spend, tokens, and tool concurrency. A run can stay under a step limit yet spend too long in one tool or consume excessive output. Also detect non-progress: repeated equivalent calls, unchanged evidence, or the same error cycle. Budget exhaustion should produce a clear terminal or paused state with useful partial artifacts.

Define success using external evidence. A research task needs cited support; a coding task needs an inspectable patch and meaningful verification; a booking task needs a durable receipt. “The model says done” is not enough. Stop when the completion evidence is met, when an unrecoverable condition appears, or when budget is exhausted, and make those reasons visible to the caller.

  • Approval pauses are durable states
  • Replanning follows changed observations
  • Multiple agents add coordination and context cost

worked example

Choose a workflow before choosing an agent loop

A document intake process always extracts fields, validates them, and routes exceptions. A separate research task must choose what evidence to inspect next.

  1. Represent intake as a fixed workflow with a model extraction step. The order and allowed actions are known in advance.
  2. Use a bounded agent only where observations must change the next action, as in selecting the next research source.
  3. Persist run state, tool results, budgets, and checkpoints. Stop after repeated non-progress, excessive cost, or an unmet permission requirement.
  4. An adaptive loop can propose a new action, but it cannot expand its own tool permissions or bypass approval.

Design decision: Use adaptive control for genuine task uncertainty and keep known sequencing deterministic.

Limit: A fixed workflow can still have probabilistic model steps, and an agent can contain deterministic subworkflows. The boundary is about control, not branding.

Teaching scenario · traffic, prices, and capacity values are illustrative.

lab

interactive lab

Agent Loop Console — healthy lifecycle vs oscillation

1 · predict first

How many identical failed tool calls before you would halt the loop? Pick your number, then run the faulty trace.

A guess makes the comparison useful. This lab draft lasts until you leave the page.

pedagogical model — values are illustrative, not measured.

assignment

Produce: Two workflow diagrams and a bounded agent state machine with stop and recovery rules.

Paper, a text editor, or a spreadsheet is enough. Model access and paid services are optional.

  1. Classify intake, research, code review, and invoice approval by what truly requires adaptive next-action choice.
  2. Use the lab to identify a healthy trace and oscillation; write the state needed to recover each.
  3. Choose budgets for steps, time, cost, retries, and non-progress; trace a tool error and a request for an unauthorized action.

Change a constraint

  • The agent alternates between the same two tools without changing evidence.
  • The process restarts after a side effect but before recording its result.

Self-review rubric

  • Adaptive behavior has a specific justification.
  • Budgets terminate an unproductive run.
  • Resume behavior preserves effect identity and permissions.

If a criterion is missing, revise that part of the artifact before rating your practice below.

Compare a sample artifact after your attempt

One defensible answer under these assumptions. Compare the reasoning and checks with your work; other designs can satisfy the same contract.

Document intake follows known steps. Research has an uncertain evidence path. Example budgets below are exercise policy choices, not universal limits.

Intake: extract → validate → route or human exception queue
Research: inspect goal → choose permitted read → observe → update evidence
          → verify coverage → finish or bounded next step
Durable run: input/version, goal, steps, tool attempts, evidence IDs, budget
Agent stop: supported artifact OR 8 steps OR 2 repeated no-progress calls
           OR 60 s OR spend cap → completed/partial/failed
Restart: restore state → reconcile uncertain effects → resume safe step

A model inside extraction does not make the surrounding fixed process an agent. Use adaptive control flow only for the uncertain research path. Budgets include per-tool deadlines and concurrency. Repeated calls without changed evidence trigger a stop or a bounded new strategy, not indefinite self-reflection.

Check your artifact

  • Both diagrams identify who chooses the next step.
  • State needed for restart lives outside the conversation.
  • Completion requires an artifact and evidence; a worker crash after an effect cannot simply replay the last instruction.

A tempting mistake: A step limit alone does not bound wall time, spend, or a single runaway tool invocation.

cumulative practice

Harbor · build and evaluate a support assistant

Project brief

Keep one project document and a folder of fictional input/output fixtures for Harbor, a workspace-scoped support assistant. Start with ticket triage and policy-backed drafts at 20 requests/minute, 1,500 input and 200 output tokens/request. Human users remain responsible for consequential actions until you design explicit permissions and approval. Later chapters add retrieval, controlled tools, durable runs, evaluation, security, and scale. Use paper, a text editor, and mocked results; paid APIs are optional. Preserve first attempts, evidence, and revisions within the existing practice budget.

Keep one project document across this track. Reconstruct earlier reasoning before opening notes, then revise the same artifact. Use part of the assignment and review budget for this work. If you enter mid-course, sketch the earlier decisions first using the linked chapters.

Add to your project: Harbor's fixed support workflow and one bounded investigation branch.

Recall the authorized tool lifecycle. Keep classify → retrieve policy → draft → validate as a workflow. Add an optional read-only investigation for missing evidence, with a durable state record, progress detector, and stop rule. Explain why refund execution remains separately controlled.

Compare cumulative reasoning after your attempt

The outer workflow owns permissions, operation identity, deadlines, and publication. A read-only branch may choose which permitted evidence to inspect within a limited budget, then return evidence IDs or partial failure. Repeated equivalent reads without new evidence stop. It cannot grant itself a refund tool or bypass exact approval. Recovery restores durable steps and reconciles effects before resuming.

Keep your first attempt and revision together. If your answer differs, identify the assumption or mechanism that explains the difference before rating your recall.

senior signal

Define the agent's success condition and maximum autonomy before describing the planner.

failure patterns

  • Using an agent for a fixed three-step API workflow
  • Keeping all state only in prompt history
  • No non-progress or cost limit
  • Multi-agent design with no independent work or ownership boundaries

knowledge checks

Start with these quick checks, then explain the closed-book questions in your own words.

  1. 01When is a deterministic workflow preferable?

    select an option to check your answer

  2. 02What should survive worker loss?

    select an option to check your answer

closed-book recall

Close the worked example and explain these from memory. Answers are self-reviewed. Drafts stay on this page; save anything you want to keep in chapter notes.

Rate the assignment and recall together

Use the assignment rubric as evidence. Completion is your own judgment; this score schedules practice.

interview drill

Design a web research agent that may run 30 minutes and produce a cited report.

  • 30 min
  • web tools
  • citations
  • partial success
compare a strong approach after your attempt

Persist a research plan, claims, source records, and checkpoints. Search in bounded branches, deduplicate sources, validate citation-to-claim coverage, and track budget/non-progress. Treat pages as untrusted data, require stronger evidence for high-stakes claims, and finish with either a coverage-checked report or explicit gaps.

Compare decisions and failure behavior, then repeat with a changed constraint. Keep your own wording.

Use the timed mock workspace →

references

Sources offer deeper context. Provider capabilities and API contracts can change; check current official docs before implementing.

notes

saved locally