CCA-F Domain 4 Study Guide: Prompt Engineering & Structured Output

There is no single "structured output" switch. There is a reliability ladder, and the exam wants you to select the rung that matches the strictness the downstream system demands. A dashboard that tolerates an occasional malformed row can live near the bottom of the ladder; a billing pipeline that inserts directly into a typed database column needs the top.

From strongest schema guarantee to weakest:

1. 01Strict tool use ("strict": true on the tool definition) — Anthropic compiles your JSON schema into a grammar and constrains sampling to it, so the emitted tool_use.input is guaranteed to validate against the schema. This is the strongest compliance available and the right answer whenever a parser downstream will crash on anything unexpected.
2. 02Structured outputs on the response (output_config.format with a json_schema) — constrains the model's text response to your JSON schema rather than a tool call. Use it when you want the answer itself to be schema-valid JSON, not a function argument.
3. 03Tool use with a JSON schema, non-strict — the schema strongly steers the model and you still parse tool_use.input, but without grammar constraints an unusual value can occasionally slip through. Validate on receipt.
4. 04Prompt-based "return JSON" — you describe the shape in the prompt and hope. Weakest: the model may add prose, a code fence, or a trailing comment. Only acceptable when a human reads the output or a lenient parser cleans it up.

The exam trap: equating any JSON output with guaranteed schema compliance. "We asked for JSON and usually get JSON" is prompt-based reliability. If the scenario says the result feeds a strict typed consumer, only strict tool use (or structured outputs) actually guarantees the contract.

A schema that requires every field forces the model to invent values when a field is genuinely absent from the document. The fix is to design the schema so "not present" and "unsure" are representable states, not error conditions the model has to route around by hallucinating.

• Optional fields — leave truly-optional keys out of required so their absence is legal.
• Nullable values — allow null (via "type": ["string", "null"] or an anyOf with a null branch) so the model has an explicit way to say "this field exists in the schema but not in this document."
• Enums for closed sets — constrain fields with a fixed vocabulary (currency, document type, status) to an enum, and include an explicit "unknown" member so ambiguity has a home instead of becoming a fabricated guess.

{
  "type": "object",
  "properties": {
    "invoice_number": { "type": ["string", "null"] },
    "issue_date":     { "type": ["string", "null"], "format": "date" },
    "currency":       { "type": "string", "enum": ["USD", "EUR", "GBP", "unknown"] },
    "total_amount":   { "type": ["number", "null"] },
    "vendor_name":    { "type": ["string", "null"] },
    "line_items": {
      "type": "array",
      "items": {
        "type": "object",
        "properties": {
          "description": { "type": "string" },
          "amount":      { "type": ["number", "null"] }
        },
        "required": ["description", "amount"],
        "additionalProperties": false
      }
    }
  },
  "required": ["invoice_number", "currency", "total_amount", "line_items"],
  "additionalProperties": false
}

Note a structured-outputs constraint to remember: objects need "additionalProperties": false, and each object lists its required keys. String/number range constraints (minLength, minimum) and recursive schemas are not enforced by the grammar — validate those client-side.

The exam trap: marking every field required and non-nullable "to make sure we get complete data." That does the opposite — it guarantees the model fills blanks with plausible fiction. Optionality is an accuracy feature, not a looseness.

Two independent levers appear in D4 scenarios and are easy to confuse. A JSON schema on the tool (input_schema, optionally "strict": true) governs the shape of the arguments when a tool is called. `tool_choice` governs whether a tool is called at all. A downstream failure like "the model replied with a friendly paragraph instead of calling record_extraction" is a tool_choice problem, not a schema problem.

{
  "tools": [{
    "name": "record_invoice",
    "description": "Record the fields extracted from an invoice document.",
    "strict": true,
    "input_schema": {
      "type": "object",
      "properties": {
        "invoice_number": { "type": ["string", "null"] },
        "total_amount":   { "type": ["number", "null"] },
        "currency":       { "type": "string", "enum": ["USD", "EUR", "GBP", "unknown"] }
      },
      "required": ["invoice_number", "total_amount", "currency"],
      "additionalProperties": false
    }
  }],
  "tool_choice": { "type": "tool", "name": "record_invoice" }
}

With strict: true the arguments are grammar-constrained to the schema; with tool_choice naming the tool, the model must emit those arguments rather than a sentence. Together they turn "extract this invoice" into a call whose input is guaranteed to be valid, parseable, and present.

The exam trap: reaching for a better schema when the real problem is that the model sometimes doesn't call the tool. A schema can't fix a missing call — only tool_choice guarantees invocation.

Real documents are inconsistent: dates as 03/04/2026 or 4 March 2026, amounts as 1,234.50 or $1.234,50, vendor names in headers or footers. Three patterns, stacked, drive accuracy and consistency up while pushing hallucination down.

1. 01Structured schema with optional/nullable fields (objective 21) — so the model reports absence instead of inventing it.
2. 02Format-normalization instructions — tell the model the canonical output form: "Return all dates as ISO 8601 (YYYY-MM-DD). Return amounts as plain decimal numbers with a period decimal separator and no currency symbol or thousands separators." This collapses many input formats into one predictable output.
3. 03Few-shot examples — one or two worked input→output pairs anchor edge cases (a missing field returning null, a European-format amount normalized) far more effectively than prose describing them.

[
  { "role": "user", "content": "Extract from: 'Rechnung 88-A, Betrag 1.234,50 EUR, Datum 04.03.2026'" },
  { "role": "assistant", "content": "{\"invoice_number\": \"88-A\", \"total_amount\": 1234.50, \"currency\": \"EUR\", \"issue_date\": \"2026-03-04\"}" },
  { "role": "user", "content": "Extract from: 'Invoice 90-B, total blank, dated March 5 2026'" },
  { "role": "assistant", "content": "{\"invoice_number\": \"90-B\", \"total_amount\": null, \"currency\": \"unknown\", \"issue_date\": \"2026-03-05\"}" }
]

The exam trap: assuming a bigger, sterner prompt ("BE ACCURATE. DO NOT MAKE MISTAKES.") raises extraction quality. It doesn't. Concrete normalization rules plus one or two positive examples outperform emphatic adjectives every time.

Extraction quality is not a one-time prompt-tuning event; it is a loop. The design objective is to capture structured metadata about each error so that patterns — not anecdotes — drive the next revision of the prompt, schema, or few-shot set.

Log, per failed or corrected extraction, a small record you can aggregate:

{
  "document_id": "inv-2026-0442",
  "field": "total_amount",
  "error_type": "fabricated",
  "model_value": 1250.00,
  "correct_value": null,
  "document_format": "scanned_pdf_eu",
  "root_cause_hypothesis": "amount absent; schema lacked null option"
}

The loop is: collect structured error metadata → aggregate to find the dominant error_type/document_format cluster → change the one lever that pattern implicates → re-run against the same evaluation set → confirm the cluster shrank. Because each error carries field, error_type, and document_format, you fix categories rather than chasing individual documents.

The exam trap: treating logging as sufficient. Raw transcripts alone don't close the loop — you need the structured signal (error type, field, format) that tells you which of prompt, schema, or examples to change.

The final objective is a processing-mode decision driven by three questions: How latency-sensitive is the caller? Does the workflow block on the result? And is a longer, less predictable processing window acceptable?

• Synchronous Messages API — you send a request and wait for the response inline. Interactive, low-latency, blocking. The right choice for chat, live extraction where a user is waiting, or any step whose result the very next line of code needs.
• Asynchronous Message Batches API — you submit up to many thousands of requests at once, the batch processes in the background, and you poll its status (in_progress → ended) and collect results when done. Non-blocking and built for high-volume, latency-tolerant work.

Anthropic documents the Batches API as offering a 50% cost reduction on token usage, with batches processing within a window of up to 24 hours (most complete sooner) and results retrievable once the batch's status reaches ended. Results come back keyed by the custom_id you assigned each request — and in any order — so always match by id, never by position.

The exam trap: choosing Batches purely because it is cheaper, in a scenario where a user is waiting. The 50% discount is real, but batch turnaround is measured in the processing window, not milliseconds — using it behind an interactive request breaks the experience. Match the mode to the blocking behavior first, then enjoy the savings where they fit.