CCAR-P Domain 6: Stakeholder Communication & Lifecycle Management

The single most valuable architect behavior in this domain mirrors how a good agent behaves: ask before assuming. When a stakeholder says 'we want an AI that answers customer emails,' the request underdetermines the design. Which email categories? What is the cost of a wrong answer versus a deflected one? Who owns the reply legally? An architect who jumps straight to a solution inherits every unstated assumption as future rework. Structured elicitation — interviews, observation of the real workflow, and written confirmation of scope — is the cheapest risk reduction available.

Treat elicitation as an explicit phase with artifacts, not a hallway conversation. The core outputs are a problem statement, success criteria that are measurable, named constraints (budget, latency, compliance, data residency), and an explicit list of non-goals. Non-goals matter as much as goals: they are how you prevent scope creep and set expectations about what the system will deliberately not attempt.

Enterprise architects translate. A board member does not want to hear about context windows and token throughput; they want to know what it costs, how fast it is, how often it is wrong, and what the worst realistic failure looks like. Frame every significant decision as a tradeoff among four axes — cost, latency, reliability, and risk — and make the tension explicit. Almost no AI decision improves all four at once; pretending otherwise erodes trust when reality arrives.

• Cost: per-request and monthly spend, including retries, longer prompts, and human review. Prompt caching and model right-sizing (a smaller model for easy cases) are the main levers.
• Latency: p50 and p95 response time, and whether streaming makes a slow response feel acceptable to the end user.
• Reliability: how consistently the system produces an acceptable answer, expressed as a rate over a representative sample, never as an absolute guarantee.
• Risk: the blast radius of a bad output — a wrong summary versus an unauthorized refund — and what controls (human-in-the-loop, confidence gating) bound it.

Use concrete, decision-ready framing: 'Option A is 40 percent cheaper but adds 600 ms and needs human review on low-confidence cases; Option B is faster and more autonomous but triples cost.' Give a recommendation with a rationale rather than dumping options. Executives are hiring your judgment, not asking you to relocate the decision back onto them.

Traditional SLAs assert deterministic guarantees: 99.9 percent uptime, sub-200 ms response. Those still apply to the plumbing around Claude — your API gateway, your queue, availability. They cannot apply to output correctness, because a probabilistic model produces a distribution of outputs, not a guaranteed one. The architect's job is to draw the line clearly: SLAs on the deterministic surface, SLOs (internal objectives) on the probabilistic quality, and no contractual guarantee of a specific answer.

Define quality objectives against a maintained evaluation set with a human-agreed grading rubric, and report them as trends over time. Pair every quality SLO with an error budget: if the acceptable-answer rate drops below target, that triggers a defined response (roll back a prompt, add review, pause a rollout) rather than an ad-hoc scramble.

Prompts and agent configurations are production code and must be versioned, reviewed, and released like code. A one-word prompt edit can shift behavior across every downstream case, so 'someone tweaked the prompt in the console' is an incident waiting to happen. Store prompts in source control, tie each version to an identifier, and never let the live system reference an unpinned or mutable prompt during a controlled release.

• Pin model versions explicitly; do not silently ride 'latest,' because a model upgrade is a behavior change that must go through evaluation.
• Gate every prompt or model change behind the evaluation suite — regression-test against the graded example set before release, not by eyeballing a few outputs.
• Keep a changelog mapping each version to what changed and why, so a postmortem can trace behavior to a specific version.
• Make rollback trivial: the previous prompt and model pin should be one config change away.

Because you cannot fully predict how a model or prompt change behaves on live traffic, roll changes out progressively and watch real metrics. Canary, staged, and feature-flag strategies limit blast radius and give you a clean rollback boundary. Shadow (mirror) deployment is especially powerful for AI: run the new version against real traffic without serving its outputs, compare quality offline, and promote only when the evaluation holds.

Deprecation is the mirror image and is frequently under-planned. When retiring an old prompt, model, or endpoint, announce a timeline, provide a migration path with overlap where both versions run, and communicate to every consumer — including internal teams who wired against it. For upstream model deprecations (a provider retiring a model version), the architect owns detecting the notice, evaluating the successor, and migrating before the cutoff rather than being surprised by it.

Operating an AI system responsibly means the on-call engineer at 3 a.m. can act without you. That requires runbooks: how to disable the feature via its flag, how to roll back to the previous prompt and model pin, how to read the quality and safety dashboards, and who to escalate to. Document the system's known failure modes and their signatures so responders recognize a hallucination spike versus an upstream outage versus a cost runaway.

Incident response for AI adds categories a normal SRE runbook lacks: degraded output quality (the service is up but answers got worse), a safety or policy breach, a prompt-injection exploit, and cost anomalies from runaway loops or oversized contexts. Each needs a detection signal and a defined first action — often 'flip the kill switch and fall back to a safe default or human path' while you diagnose.

Postmortems must be blameless and focus on systemic causes. For AI incidents the useful questions are: which prompt/model version was live, what did the evaluation set miss, was there a guardrail gap, and what regression test or monitor would have caught it. Convert every postmortem into a concrete addition to the eval suite or the monitoring, so the same class of failure is caught automatically next time.

AI initiatives fail most often at the seams between phases. A POC optimizes for 'is this possible?' and can cut every corner — hardcoded prompts, no evals, a hand-picked demo dataset. Promoting that POC directly to production is the most common and most expensive mistake in the domain, because the demo never faced the long tail of real inputs, adversarial users, cost at scale, or the operational tooling production demands.

Maintenance is not idle. Model behavior effectively drifts as providers update models and as real-world inputs shift away from what the prompt was tuned for, so quality must be re-measured on a schedule, not assumed constant. Budget explicitly for this ongoing operation; a system with no owner and no recurring eval is technical debt accruing silently until it fails a user.