CClaude Cert Prep
P4 · 8 questions

Evaluation, Testing & Optimization — Attempt 1

CCAR-P P4 practice. Some items are “select all that apply” — they tell you how many to pick and are graded all-or-nothing. Commit before revealing, then read every explanation.

0 of 8 answered
  1. P4 · Q1eval-driven development
    A team is about to start building a customer-support summarization feature. Before writing any prompts, what is the most appropriate first step under an eval-driven development approach?
  2. P4 · Q2RAG eval — retrieval vs faithfulness
    A RAG system gives wrong answers. To diagnose the failure correctly, what is the best way to structure the evaluation?
  3. P4 · Q3bulk eval cost
    You need to run an LLM-as-judge evaluation over 200,000 stored transcripts. Results are needed for a weekly quality report, not immediately. Which approach is most appropriate?
  4. P4 · Q4LLM-as-judge — position bias
    An LLM judge compares two candidate answers (A vs B) and picks the better one. In an audit, the judge favors whichever answer is presented first about 65% of the time, regardless of content. Which mitigation most directly addresses this?
  5. P4 · Q5cost-optimization levers
    A high-traffic FAQ assistant repeatedly answers the same top 50 questions with nearly identical prompts. Latency is fine, but per-request cost is too high. Considering the traffic pattern, which lever is most appropriate to apply first?
  6. P4 · Q6quality-cost-latency trade-off
    A medical-intake triage assistant must meet a hard contractual requirement of factual accuracy on a validated eval set. Under budget pressure, the team proposes several changes. Every option lowers cost or latency, but which is the least risky choice given the hard accuracy constraint?
  7. P4 · Q7eval dataset designSelect 2
    You are designing a golden eval set for a contract-clause classifier. Which two practices most improve the set's ability to catch real-world failures?
  8. P4 · Q8regression testing in CISelect 2
    Prompt changes keep causing silent quality drops that reach production before anyone notices. Which two practices best turn this into a caught-in-CI regression problem?
More P4 attempts