CCAR-P Domain 3: Integration — RAG, MCP, Auth & Observability

The first architectural decision is whether you need retrieval at all. Claude's large context windows mean that for a bounded corpus — a single contract, a policy handbook, a few dozen pages — you can often place the whole document in the prompt and skip the entire RAG pipeline. RAG earns its complexity when the knowledge base is too large to fit, changes frequently, requires access controls per document, or when you need citations traceable to specific sources. Fine-tuning is not an option to reach for on this exam: Anthropic does not offer customer fine-tuning of Claude models, so any answer proposing 'fine-tune the model on the knowledge base' is wrong by construction. The real trade-off is retrieval versus long-context, and you make it on corpus size, freshness, access control, and cost-per-call.

A common hybrid the exam likes: use RAG to narrow a huge corpus down to a handful of candidate documents, then place those full documents in the context window rather than tiny fragments. This preserves the grounding of retrieval while giving Claude enough surrounding text to reason well — retrieval for selection, long-context for comprehension.

A RAG pipeline has three quality-determining stages, and architects are tested on knowing which stage a symptom points to. Chunking decides what a retrievable unit is. Retrieval (usually vector similarity, often combined with keyword/BM25 as hybrid search) decides which units are candidates. Re-ranking re-scores those candidates with a stronger model to put the truly relevant ones on top. A frequent trap: the answer improves recall by retrieving more chunks, but that pushes precision down and floods the context with noise. The architecture-grade fix for 'right document exists but ranks low' is a re-ranking stage, not simply raising top-k.

• Chunk on semantic boundaries (sections, headings), not fixed character counts that split sentences mid-thought.
• Add overlap between chunks so a fact spanning a boundary is not lost; too much overlap wastes storage and blurs retrieval.
• Store metadata (source, section, timestamp, ACL) alongside each chunk — you filter and cite on it later.
• Use hybrid retrieval (semantic + lexical) so exact identifiers, error codes, and names are not lost by pure embeddings.
• Add a re-ranker over the top candidates to fix ordering; retrieve broad, then re-rank narrow.
• Consider contextual retrieval — prepend a short chunk-specific summary before embedding so isolated chunks retrieve on their true meaning.

Evaluate the two stages separately. Retrieval quality is measured with metrics like recall@k and precision@k against a labeled set of question-to-passage pairs; answer quality is measured on the generated output. Conflating them hides where a regression lives — a good answer can mask poor retrieval when the model already knew the fact, and a bad answer can come from perfect retrieval poorly synthesized.

Retrieval is only useful if the model actually grounds its answer in the retrieved text and can point to where each claim came from. At enterprise scale, citations are not a nicety — they are how reviewers audit answers, how you detect retrieval drift, and how legal accepts the system at all. Design the prompt so Claude cites the specific source or chunk id for each assertion, and so it is explicitly permitted to say the answer is not in the provided context rather than filling the gap from parametric memory.

• Tag each retrieved passage with a stable identifier the model can cite verbatim.
• Instruct the model to answer only from provided context and to abstain when the context is insufficient — abstention is a feature, not a failure.
• Surface citations in the response so downstream review and evaluation can check that quoted spans actually support the claim.
• Log retrieved-chunk ids with each answer so a wrong answer can be traced to whether retrieval or synthesis failed.

The Model Context Protocol exposes three distinct primitives, and choosing the right one is a recurring exam theme. Tools are model-controlled actions — the model decides when to invoke them, and they can have side effects (send an email, query a database, create a ticket). Resources are application-controlled context — addressable read-only data (a file, a record, a schema) the host application attaches to context. Prompts are user-controlled templates — reusable, parameterized interactions a user deliberately selects. The mistake is modeling everything as a tool; read-only reference data belongs as a resource, and a canned workflow a user triggers belongs as a prompt.

At scale the dominant failure is tool-surface bloat. When a single server exposes dozens of overlapping tools, the model's tool-selection accuracy drops and token cost per call rises. The architectural remedies are consolidation (fewer, well-scoped tools with clear, non-overlapping purposes), precise descriptions and schemas so the model can disambiguate, and segmentation of tools behind scopes so a given session only sees the tools its role needs. Treat the tool list as an API surface you version and curate, not a junk drawer.

MCP defines two standard transports. stdio is for local servers the host launches as a subprocess — lowest latency, no network exposure, ideal for developer machines and desktop tooling. Streamable HTTP is for remote servers reached over the network — the right choice for a centrally hosted, multi-tenant enterprise server, and the transport that participates in the OAuth authorization flow. Choosing stdio for a service that must serve many users centrally, or HTTP for a purely local dev tool, is a classic mismatched-transport trap.

Tool schemas are a contract. Each tool declares a JSON Schema input, and the server should validate against it and return structured, actionable errors rather than free-text or silent failures. A tool that returns a 200 with an error buried in prose teaches the model nothing about how to recover; a tool that returns an explicit error result with a machine-readable reason lets the model retry, adjust arguments, or surface the failure cleanly.

{
  "name": "lookup_invoice",
  "description": "Fetch a single invoice by its exact id. Read-only.",
  "inputSchema": {
    "type": "object",
    "properties": {
      "invoiceId": { "type": "string", "description": "Exact invoice id, e.g. INV-10432" }
    },
    "required": ["invoiceId"]
  }
}

// A well-formed error result the model can act on:
{
  "isError": true,
  "content": [{ "type": "text", "text": "NOT_FOUND: no invoice with id INV-99999. Verify the id and retry." }]
}

Identity is the highest-stakes seam in an integration. A remote MCP server serving multiple users must carry each user's identity through to the downstream system so that access checks and audit logs reflect the actual person — not a single shared service account. The exam's canonical anti-pattern is a shared static token or service credential used for every user: it collapses per-user authorization, makes least-privilege impossible, and turns one leaked secret into total compromise. The correct pattern is OAuth-based, per-user tokens with the narrowest scopes the task requires.

• Use OAuth 2.1 for remote MCP servers so each user authenticates and the server acts with that user's authority, not a global one.
• Scope tokens to least privilege — a read-only reporting agent should never hold write or admin scopes 'just in case'.
• Never hardcode secrets in server code or config; inject them via environment variables / a secrets manager, and reference them by expansion (e.g. ${DB_PASSWORD}) so the value never lives in the repo.
• Rotate and revoke: per-user, short-lived tokens contain a breach; a long-lived shared key turns any leak into a full compromise.
• Keep authorization server-side — never trust the client to tell you what it is allowed to do.

Secrets hygiene extends to configuration. Environment-variable expansion in MCP/client config lets you reference a secret by name so the actual value stays in a vault or the deployment environment, never checked into source control. If a scenario shows a password literal sitting in a committed config file, the finding is 'move it to an env var / secrets manager and expand by reference,' regardless of what else is going on.

You cannot operate what you cannot see. An agent run is a tree of steps — model turns, tool calls, retrievals — and the unit of observability is the trace that ties them together under one correlation id. For each step you want inputs, outputs, token counts (input, output, and cache-read/write), latency, and the tool result or error. Without per-step attribution, a cost spike or latency regression is unattributable, and 'the agent got slow' is as far as anyone can diagnose.

Evaluating retrieval quality in production is its own discipline. Offline metrics drift as the corpus and query mix change, so architects sample real production queries, score retrieval against a maintained labeled set, and gate deploys on those metrics. Prompt caching adds a wrinkle worth tracking: cache-read tokens are far cheaper than uncached input, so cost dashboards should separate cache hits from misses — a rising bill can be a falling cache-hit rate, not more traffic.

Every external dependency — an MCP tool, a retrieval backend, a downstream API — will eventually be slow or down, and the architecture has to degrade rather than collapse. The core patterns: bounded timeouts so no call hangs indefinitely; retries with exponential backoff and jitter for transient failures, but only on idempotent operations; and circuit breakers that stop hammering a failing dependency and fail fast until it recovers. Retrying a non-idempotent action (charge a card, send an email) is a classic wrong answer — it duplicates side effects.

• Set explicit timeouts on every tool and retrieval call; an unbounded call is a latent outage.
• Retry only idempotent operations, with exponential backoff and jitter to avoid synchronized retry storms.
• Wrap flaky dependencies in a circuit breaker so repeated failures fail fast instead of piling up latency.
• Return structured errors the model (and your logs) can act on, so a partial failure degrades gracefully instead of poisoning the whole run.
• Prefer graceful degradation — a cached or partial answer with a clear caveat beats a hung request or a crash.

These patterns compose with observability: a circuit breaker's state transitions, timeout trips, and retry counts are themselves telemetry. When the exam asks how to keep an agent responsive while a dependency flaps, the complete answer pairs a reliability pattern (timeout, breaker) with the signal that makes it visible (log the trip, alert on the open breaker).