Insurance · 13 linked questions

The Adjuster's Second Pair of Hands: Auditable AI at Meridian Mutual

A mid-to-large personal-lines insurer wires Claude into its claims desk to draft adjuster replies to policyholders. High volume, tight SLAs, and a state regulator that can demand proof of every automated decision force the architecture to earn its keep on evidence, not just accuracy.

One evolving scenario Questions build on the context Rate your confidence

How to use it · Read the brief, then work top to bottom — the situation evolves as you go. Rate your confidence before submitting; on review, the “ confident & wrong” questions are the misconceptions worth fixing first. A study tool, not the graded exam.

The scenario

Meridian Mutual handles roughly 8,000 inbound policyholder claim queries a day across auto and homeowners lines. Leadership wants Claude to draft adjuster responses so licensed adjusters can review-and-send instead of writing from scratch, cutting handle time without ceding the licensed judgment the state requires. Every automated touch has to survive a market-conduct examination years later.

Architecture: Claude via API behind an internal 'Draft Assist' service; retrieval over policy documents, coverage rules, and claim notes; drafts are proposals surfaced in the adjuster console, never auto-sent to policyholders.
Existing stack: Guidewire-style claims system of record, a document store for policy forms, an existing case-management queue, and a SIEM the compliance team already trusts for tamper-evident logs.
Compliance: State insurance regulation (unfair-claims-practices statutes) with a records-retention and audit-trail obligation: the insurer must reproduce, on demand, what was communicated, on what basis, and who was accountable — retained for the statutory period (7 years).
Volume: ~8,000 queries/day steady-state, spiky around catastrophe events; ~240k/month.
Budget: Approved run-rate assumes review-and-send drafting only; no budget line for model fine-tuning or a dedicated ML training team.
SLA: Median first-response to policyholder within 4 business hours; high-value and coverage-denial claims carry stricter internal timers and mandatory licensed sign-off.

Keep this context in mind — later questions build on it, and the situation evolves as you go.

0 of 13 answered

P6 · Q1Load-bearing control: per-interaction audit trail
The program lead asks which control most directly satisfies the state's audit-trail obligation for automated claim communications. Which is the MOST appropriate primary control to make load-bearing?
How confident are you?
P4 · Q2Escalation on policy, not model self-confidence
Adjusters want low-risk claims to flow faster. An engineer proposes: 'Have Claude output a confidence score, and auto-clear any draft where Claude reports high confidence.' What is the LEAST risky design?
How confident are you?
P1 · Q3No fine-tuning of Claude; retrieval as the fit mechanism
To make drafts match Meridian's coverage language, a stakeholder pushes to fine-tune the model on five years of closed claim files. Given the budget and stack, what should the architect recommend?
How confident are you?
The situation changes
Segment 1 — The market-conduct examiner calls
Nine months in, the state's Department of Insurance opens a routine market-conduct examination. The examiner requests evidence of controls: for a specific list of 40 claim numbers, produce what was communicated to each policyholder, the basis for it, whether AI assisted, and who approved the outgoing message. Meridian has 15 business days to respond. The Draft Assist logs are the first thing counsel reaches for.
P6 · Q4Reconstructing a decision from the audit trail
For the 40 requested claims, counsel needs to reconstruct each interaction end to end. Which record set MOST completely satisfies the examiner's request?
How confident are you?
P5 · Q5Gains / Gives-up / Reversal-cost of a logging shortcutSelect 2
During the exam, the team discovers that to save storage the retrieved policy snapshot was NOT persisted — only a pointer to the 'current' policy document was kept, and those documents have since been updated. Using a Gains / Gives-up / Reversal-cost lens, which TWO statements are correct?
How confident are you?
P4 · Q6Must-hold rule in deterministic code, not a prompt
State rules forbid the insurer from making coverage-denial statements without a licensed adjuster's sign-off. The team must guarantee no Draft Assist output containing a denial ever reaches a policyholder unreviewed. Where should this guarantee live?
How confident are you?
P2 · Q7Grounding drafts in retrieved policy vs. model recall
An examiner samples a draft that quoted a homeowners deductible that turned out to be wrong for that policy. Root cause: the coverage figure came from the model's general knowledge, not the policyholder's actual policy. What is the MOST appropriate fix?
How confident are you?
The situation changes
Segment 2 — Catastrophe surge doubles the volume
A regional hail event doubles inbound queries to ~16,000/day for several weeks. The SLA timers keep running, and executives are tempted to loosen the human-review workflow to keep pace. The examination from Segment 1 closed with no findings — but the examiner explicitly praised the per-interaction trail, so the controls are now something to protect, not weaken.
P5 · Q8Scaling throughput without weakening controls
To hold the 4-hour SLA during the surge, which approach best increases throughput while keeping the controls the examiner praised intact?
How confident are you?
P6 · Q9Outcome document: an auditable control with a measurement ownerSelect 2
Compliance asks for an 'outcome document' proving the review workflow held during the surge. Which TWO elements make it a genuine auditable control rather than a status update?
How confident are you?
P1 · Q10Cost control under surge without new budget
Finance flags that surge volume is straining the approved run-rate. Given the budget excludes fine-tuning and training staff, which is the MOST appropriate cost lever that doesn't compromise controls?
How confident are you?
The situation changes
Segment 3 — Legal mandates human sign-off for claims over $25,000
After the surge, Legal issues a firm rule: no communication that concedes, denies, or estimates on any claim with exposure over $25,000 may leave the building without a licensed adjuster's recorded sign-off — no exceptions, no thresholds that move under load. Engineering must make this binding and prove it stays bound.
P6 · Q11Binding the $25k sign-off as an enforced invariant
Where and how should the new $25,000 sign-off mandate be implemented so it is both binding and provable?
How confident are you?
P4 · Q12Eval as acceptance criteria before the gate shipsSelect 2
Before the $25,000 gate goes live, the team wants evals that act as acceptance criteria. Which TWO evals most directly qualify the gate for release?
How confident are you?
P7 · Q13Sustaining and governing the control over time
Six months on, coverage forms change and the exposure field's source system is being migrated. What is the MOST appropriate way to keep the audit-trail and sign-off controls trustworthy through change?
How confident are you?

All case studies