01Map prompts to dominant difficulty
Feeds are fan-out and ranking. Chat is ordered low-latency delivery plus offline sync. File storage is metadata plus immutable blobs. Autocomplete is prefix retrieval under a tight latency budget. Payments are ledgers and idempotent state transitions.
Classifying a prompt means finding its invariant and dominant access pattern, not matching it to a memorized diagram. A feed emphasizes fan-out and freshness; chat emphasizes conversation ordering and delivery; file storage separates metadata from large immutable bodies. Start with one representative request and ask which part becomes difficult under the supplied scale and failure constraints.
Compare two feed accounts. An author with 200 followers can fan out new posts to follower inboxes cheaply. An author with 20M followers creates an expensive write burst. A hybrid can precompute ordinary authors and merge celebrity posts at read time. It adds read complexity and freshness rules, but targets the actual skew. “Use fan-out on write” is incomplete without the follower distribution.
Reuse reasoning patterns across prompts: stable operation IDs for retries, bounded queues for overload, authorization at data access, and measured triggers for scaling. Keep the product-specific semantic question visible. Duplicating a recommendation is annoying; duplicating a money movement can break an invariant. The same infrastructure component does not imply the same acceptable failure behavior.
- Name the hardest invariant
- Name the hottest path
- Name the acceptable degradation
02Infrastructure prompts need operators
Schedulers, caches, gateways, metrics systems, and feature flags are judged through control/data plane separation, propagation behavior, tenant isolation, and operational safety.
A job scheduler has a control path for submitting and inspecting work and a data path for acquiring and executing it. Explain who owns a lease, how a stale worker is fenced, and how fairness is measured. A distributed cache similarly needs placement, membership, eviction, hot-key handling, and behavior during node loss. The operator's recovery path is part of the design.
For a metrics platform, high ingestion throughput is only one difficulty. Unbounded label combinations can create huge series counts; retention and aggregation choices affect storage and queries. For feature flags, compare a control plane that edits configuration with a serving path that evaluates a cached version. Decide what happens if updates stop propagating while application traffic continues.
Build a short comparison page after several problems. Record the core invariant, hot path, partition key, failure response, and first scaling trigger for each. If all rows say “cache, queue, database,” revisit the designs: you have recorded components instead of the mechanisms that distinguish the problems.
- Configuration rollout needs versions and rollback
- Control plane outage should not always stop data plane
- Think about cardinality and noisy neighbors
03AI prompts combine familiar patterns
LLM chat adds expensive streaming compute to messaging. Coding agents add long-running job orchestration and sandboxed tools. RAG adds search indexing, ACLs, and grounded generation. Inference APIs add admission and GPU scheduling.
Map AI prompts to familiar systems before adding model-specific concerns. An LLM chat service combines conversation storage with expensive streaming compute. A coding agent combines job orchestration, repository state, tool execution, and verification. Enterprise RAG combines document ingestion, search, permissions, evidence assembly, and generation. Each still needs ordinary ownership and recovery rules.
Then identify what changes: variable output quality, token-shaped demand, model capability differences, and untrusted text influencing proposed actions. A successful HTTP response can contain a wrong answer. Track request availability separately from supported task completion, and define cost per useful result rather than only cost per model call.
Practice one prompt with different constraints instead of adding new prompts forever. Reduce the scale by ten, remove a region, revoke access mid-run, or double output length. Explain exactly which earlier decision changes. This makes transfer visible: you are adapting a mechanism to evidence rather than reciting the same architecture faster.
- Reuse known primitives; emphasize new failure modes
- Separate model quality from system reliability
- Close with evals and cost