CCAR-P Domain 2: Claude Models, Prompting & Context Engineering

The current lineup spans three practical tiers: an Opus tier (most capable, best for hard reasoning and long-horizon agentic work), a Sonnet tier (near-Opus quality on coding and agentic tasks at lower cost), and a Haiku tier (fastest and cheapest, for simple, latency-sensitive tasks). An architect's default should be the most capable tier the budget tolerates, then route down deliberately where a cheaper tier provably suffices — never the reverse, because silently downgrading to save cost is a decision the product owner should make, not a default.

A common enterprise pattern is a router: a cheap Haiku-tier classifier inspects each request and dispatches simple items to Haiku, everything else to Sonnet, and escalates only the flagged-hard cases to Opus. The router itself must be cheap and fast or it eats the savings. Measure the escalation rate — if 80% of traffic escalates to Opus, the router is theater.

Route by capability signal, not by request length. A short prompt can be a hard reasoning problem and a long prompt can be trivial extraction. Use a fast classifier, an explicit difficulty tag from the calling application, or a confidence threshold on a first cheap pass with escalation on low confidence.

Prompt caching is a prefix match. The cache key is derived from the exact bytes of the rendered prompt up to each cache_control breakpoint, and the render order is fixed: tools, then system, then messages. Any byte change anywhere in the prefix invalidates the cache for everything after it. This single invariant drives every caching decision an architect makes.

Place stable content first (a frozen system prompt, a deterministically ordered tool list, retrieved reference documents) and volatile content last (timestamps, per-request IDs, the varying user question). Interpolating datetime.now() or a request UUID into the system prompt header silently makes everything after it uncacheable, no matter how many cache_control markers you add.

# Cache a large shared prefix; keep the varying question after the breakpoint.
resp = client.messages.create(
    model="claude-opus-4-8",
    max_tokens=16000,
    system=[{
        "type": "text",
        "text": LARGE_STABLE_POLICY_DOC,        # frozen prefix, byte-identical every call
        "cache_control": {"type": "ephemeral"}   # breakpoint at end of the shared portion
    }],
    messages=[{"role": "user", "content": user_question}],  # volatile, uncached
)
print(resp.usage.cache_read_input_tokens)  # >0 on the second identical-prefix call

On the current Opus tier you can inject mid-conversation operator instructions as a role:"system" message appended to the messages array instead of editing the top-level system prompt. This preserves the cached history prefix and is the prompt-injection-safe operator channel; editing the top-level system prompt would re-process every cached turn uncached.

Current flagship models offer a 1M-token context window, but a large window is a budget, not a target. Two failure modes appear as context fills: lost-in-the-middle, where the model attends less reliably to information buried in the middle of a very long prompt than to content near the start or end; and context rot, where accumulated stale tool results, superseded plans, and completed thinking crowd out the signal the model actually needs.

• Put the most decision-relevant material near the start or the end of the prompt, not buried in the middle of a long block.
• Prune stale tool outputs as an agent loop grows rather than letting them accumulate indefinitely.
• Summarize or checkpoint completed sub-tasks so their conclusions survive but their verbose transcripts do not.
• Keep a large window in reserve as headroom for the model to think and act, rather than filling it to the brim with retrieved context.

When using compaction, the API returns a compaction block that must be passed back on subsequent requests — append the full response content, not just the extracted text, or the compaction state is silently lost and history balloons again.

On current models the recommended pattern is adaptive thinking: the model decides when and how much to think per request, and interleaves thinking between tool calls automatically. The older fixed thinking-token-budget approach is deprecated on current models and is rejected outright on the newest ones — designing a new system around a hard budget_tokens value is building on a retired API surface.

Depth and overall token spend are controlled with the effort parameter (inside output_config), not a token budget. Effort levels run low, medium, high, xhigh, max. Higher effort buys more thorough reasoning and verification but costs more tokens and latency; on the newest models, higher effort up front can actually reduce total cost on agentic work by cutting the number of turns needed.

For long agentic loops, a task budget gives the model a token ceiling it is aware of, so it paces itself and finishes gracefully instead of being cut off. This is distinct from max_tokens, which is an enforced per-response ceiling the model cannot see.

When a downstream system needs machine-parseable output, reliability is a ladder — climb to the highest rung the task supports rather than defaulting to prompt-and-hope. The rungs, from most to least reliable, constrain the output at successively weaker layers.

• Top rung: strict tool use (strict: true on the tool with additionalProperties:false + required) or output_config.format with a JSON schema — the API enforces the shape, so the output validates exactly.
• Middle rung: a non-strict tool schema — the model is strongly steered toward the shape but the guarantee is softer.
• Bottom rung: asking for JSON in the prompt and parsing the text — works, but has no enforcement and drifts under load.

# Top-rung structured output: the API guarantees the response validates.
resp = client.messages.create(
    model="claude-opus-4-8",
    max_tokens=16000,
    messages=[{"role": "user", "content": "Extract the vendor, amount, and due date."}],
    output_config={
        "format": {
            "type": "json_schema",
            "schema": {
                "type": "object",
                "properties": {
                    "vendor": {"type": "string"},
                    "amount": {"type": "number"},
                    "due_date": {"type": "string", "format": "date"}
                },
                "required": ["vendor", "amount", "due_date"],
                "additionalProperties": False
            }
        }
    },
)

Two operational notes: a brand-new schema incurs a one-time compilation cost on its first request, then caches; and structured-output format is incompatible with citations (returns a 400). Also always parse tool-call inputs with a real JSON parser rather than string-matching the serialized text — current models may escape Unicode or slashes differently than older ones.

Current models follow clear instructions closely and interpret them literally, which shifts the few-shot-vs-instructions balance. For well-specified, unambiguous tasks, precise instructions are often cheaper and more maintainable than a wall of examples. Reach for few-shot when the task is hard to describe but easy to demonstrate — an idiosyncratic format, a subtle tone, an edge-case taxonomy. When you do use examples, positive examples of the desired output tend to steer more effectively than negative do-not lists.

At scale, long-context work is managed with three structural patterns beyond raw window size. Compaction keeps a single long conversation under the limit. Scratchpads — a memory file the agent writes to and re-reads — let it externalize working state instead of holding everything in-context. Subagent isolation gives a sub-task its own fresh context window so its verbose exploration never pollutes the orchestrator's transcript, and the orchestrator receives only the distilled result.

• Compaction: summarize earlier turns server-side as the window fills; pass the compaction block back each turn.
• Scratchpads: give the agent a memory surface and tell it where to write and when to consult it.
• Subagent isolation: fan out independent sub-tasks to subagents with their own context; the parent keeps its prefix cached and its transcript clean.
• Tool search: discover tools on demand from a large library — schemas are appended, not swapped, so the prompt cache survives.