skip to content

learn.smmeyer.devllm-engineering06

menu

llm-engineering/06

application core · core · ~3h practice

Tool use & function calling

outcome

Build a safe tool lifecycle from schema exposure through validated execution and grounded continuation.

Plan a practice session · ~3h including exercises
Read + trace the worked example
25 min
Assignment + constraint variations
100 min
Timed drill + repeated explanations
35 min
Closed-book recall + self-review
20 min

This is a suggested practice budget, including drawing, variations, and repeated drills. Reading is the first block. Split the work across sessions and use the rubric to judge when to move on.

why this matters

Tools convert language decisions into real effects. The model proposes a call; your system owns availability, validation, authorization, execution, retries, and the result that returns to context.

coverage

  • Interface: tool descriptions, schemas, argument generation.
  • Lifecycle: select, call, validate, execute, return, continue.
  • Composition: multiple calls, parallel vs sequential dependencies.
  • Failure: timeouts, retries, partial results, tool selection.
  • Security: permissioning, least privilege, trusted results.
  • Autonomy: approval gates and dangerous actions.

mental model

The model proposes; deterministic infrastructure authorizes and executes.

  1. 1 →Expose

    Relevant schema

  2. 2 →Propose

    Name + args

  3. 3 →Authorize

    Validate + approve

  4. 4 →Execute

    Result + audit

Read in numbered order: expose → propose → authorize → execute.

lessons

01Schema quality drives tool quality

Names and descriptions should distinguish when tools apply; argument schemas should make invalid states difficult. Expose only relevant tools to reduce confusion and prompt footprint.

A tool call is a structured proposal, not an executed action. The application exposes a schema, receives a tool name and arguments, validates them, checks the current caller's permission, executes the allowed operation, and returns a structured observation. The model can then use that observation in its continuation. Trace these stages separately so a correct-looking proposal cannot bypass executor policy.

Use narrow names and argument contracts. “refund_order” with order_id, amount, currency, and operation_id is easier to validate than a generic “run_command” string. Explain units, required fields, and bounded values. Derive the tenant from trusted request context rather than accepting a model-selected tenant. Tool errors should identify safe retry conditions without leaking secrets or hidden resources.

  • Validate types and business rules server-side
  • Never let tool descriptions grant permission
  • Use stable call IDs for tracing and deduplication

02Parallelism follows dependencies

Independent reads can run concurrently; a call that needs an earlier result must wait. Your orchestrator should express dependency edges, set deadlines, aggregate errors, and return compact results.

Draw a dependency graph before parallelizing. Looking up independent product prices can run concurrently. Reserving stock depends on a chosen product and quantity; issuing a refund depends on an authorization decision and exact approved amount. Parallel calls are not safe merely because the provider can emit them in one response. Concurrency must preserve ordering and business invariants.

After execution, return observations tied to stable call IDs so results cannot be associated with the wrong proposal. Limit fan-out, total time, and response size. A failed tool should not automatically trigger a different tool with broader privileges. Decide which failures return a bounded observation to the model, which require deterministic recovery, and which end the run.

  • Parallel writes can conflict
  • Tool results are still untrusted data
  • Bound result size before reinserting into context

03Side effects need control

Classify tools by risk. Reads may auto-run under policy; reversible writes may need preview; destructive or external actions may require explicit human approval. Use scoped credentials and sandboxing.

Consider a refund that commits at the provider but times out before the application sees the response. A blind retry may refund twice. Bind an idempotency key to the exact business operation and parameters, persist attempts, and reconcile status before repeating an uncertain effect. If a provider offers no safe replay or status lookup, expose an unresolved state and escalation rather than claiming universal exactly-once execution.

Approval must cover the actual action. If a user approved a $20 refund and the amount later changes to $200, the old approval is not sufficient. Bind approval to the principal, resource, parameters, and relevant version; recheck permission before execution. Store an authoritative receipt separately from the model's final explanation, since a fluent success message is not proof that the provider accepted the action.

  • Retry reads more freely than non-idempotent writes
  • Return explicit errors, not invented substitutes
  • Audit actor, arguments, policy, result, and effect

worked example

A duplicated refund tool call

A model proposes refund(order_123, 50). The tool request times out after the payment provider commits, and the orchestrator retries.

  1. Treat the proposal as untrusted input. Validate the schema, currency and amount, current order state, and caller authorization.
  2. Require the applicable user approval before sending the effect; approval is tied to the specific order and amount.
  3. Use a durable operation ID as the provider idempotency key and persist the result. A retry recovers the same refund rather than creating another one.
  4. Return a structured tool result linked to the original call, distinguishing success, rejection, and an unresolved provider status.

Design decision: Make the tool executor own permissions, effect identity, and reconciliation. The model supplies a proposal.

Limit: An idempotency key only helps within the provider's defined scope and retention. An ambiguous result may still need status reconciliation.

Teaching scenario · traffic, prices, and capacity values are illustrative.

lab

interactive lab

Agent Loop Console — healthy lifecycle vs oscillation

1 · predict first

How many identical failed tool calls before you would halt the loop? Pick your number, then run the faulty trace.

A guess makes the comparison useful. This lab draft lasts until you leave the page.

pedagogical model — values are illustrative, not measured.

assignment

Produce: A tool schema, executor pseudocode, and sequence diagrams for success, denial, and timeout-after-commit.

Paper, a text editor, or a spreadsheet is enough. Model access and paid services are optional.

  1. Specify required fields, constraints, operation identity, authorization, approval, and structured errors.
  2. Step through the lab trace and identify the boundary where every check must occur.
  3. Inject a duplicate call and a stale approval; trace recovery and revise the executor contract.

Change a constraint

  • The provider offers no idempotency key or status lookup.
  • The user changes the approved refund amount before execution.

Self-review rubric

  • Permission is checked in code using the current principal.
  • Approval applies to the exact effect.
  • Duplicate delivery cannot silently create a second effect.

If a criterion is missing, revise that part of the artifact before rating your practice below.

Compare a sample artifact after your attempt

One defensible answer under these assumptions. Compare the reasoning and checks with your work; other designs can satisfy the same contract.

Refund order_123 by $50 USD. The provider supports operation-key reconciliation; amounts use integer minor units in the executor contract.

Proposal: {tool: refund, orderId: order_123, amountMinor: 5000,
           currency: USD, operationId: R1}
Executor: validate → current principal/resource authorization
          → approval bound to (order,5000,USD,R1) → persist attempt
          → provider call(key=R1) → receipt or unresolved
Success: receipt → store succeeded → return same receipt on replay
Denial: no permission/changed approval → no provider call
Timeout-after-commit: unresolved → lookup/reconcile R1 → original receipt

The model proposes; application code authorizes. Changing the amount invalidates the earlier approval. Without provider idempotency or status lookup, do not blindly replay an uncertain refund: escalate or choose an explicitly accepted duplicate/omission policy. A database flag alone cannot atomically cover the external payment.

Check your artifact

  • Replaying R1 cannot create a new business effect under the stated provider contract.
  • Reusing R1 with a changed amount is rejected.
  • Success is supported by a durable receipt, not the model's final wording.

A tempting mistake: A retry meant to repair the final response must not repeat the already committed refund.

cumulative practice

Harbor · build and evaluate a support assistant

Project brief

Keep one project document and a folder of fictional input/output fixtures for Harbor, a workspace-scoped support assistant. Start with ticket triage and policy-backed drafts at 20 requests/minute, 1,500 input and 200 output tokens/request. Human users remain responsible for consequential actions until you design explicit permissions and approval. Later chapters add retrieval, controlled tools, durable runs, evaluation, security, and scale. Use paper, a text editor, and mocked results; paid APIs are optional. Preserve first attempts, evidence, and revisions within the existing practice budget.

Keep one project document across this track. Reconstruct earlier reasoning before opening notes, then revise the same artifact. Use part of the assignment and review budget for this work. If you enter mid-course, sketch the earlier decisions first using the linked chapters.

Add to your project: Harbor's approval-gated support-action executor.

Add a fictional refund proposal path to the classifier. Reconstruct the trusted-input boundary, bind exact approval, and trace timeout-after-commit. Show how output repair differs from effect retry.

Compare cumulative reasoning after your attempt

The ticket and generated category supply no permission. Validate the proposed order, currency, amount, and stable operation identity under current user authority. Exact approval covers those parameters. Persist and reconcile uncertain provider state before replay; regenerate malformed explanatory output using the existing receipt. If the provider cannot safely reconcile, expose unresolved status instead of pretending that retry is harmless.

Keep your first attempt and revision together. If your answer differs, identify the assumption or mechanism that explains the difference before rating your recall.

senior signal

Say exactly where authority lives: the model has semantic discretion; the executor has security authority.

failure patterns

  • Executing model arguments without validation
  • Retrying a timed-out write with a new operation ID
  • Treating tool output as trusted instructions
  • Exposing every tool on every turn

knowledge checks

Start with these quick checks, then explain the closed-book questions in your own words.

  1. 01Who should make the final authorization decision?

    select an option to check your answer

  2. 02Parallel tool calls are safest when…

    select an option to check your answer

closed-book recall

Close the worked example and explain these from memory. Answers are self-reviewed. Drafts stay on this page; save anything you want to keep in chapter notes.

Rate the assignment and recall together

Use the assignment rubric as evidence. Completion is your own judgment; this score schedules practice.

interview drill

Design tool execution for an assistant that can search orders, refund purchases, and send email.

  • PII
  • money movement
  • email side effects
  • timeouts
compare a strong approach after your attempt

Separate read and write tools, use scoped identity, validate order ownership, preview refund/email, and require approval for material effects. Use idempotency keys for refund/send, log audit records, cap retries, and return explicit unknown-outcome status after ambiguous timeouts.

Compare decisions and failure behavior, then repeat with a changed constraint. Keep your own wording.

Use the timed mock workspace →

references

Sources offer deeper context. Provider capabilities and API contracts can change; check current official docs before implementing.

notes

saved locally