Trusting an LLM with a compliance decision

ScreenGPT is an LLM system we built at Chipper Cash to adjudicate sanctions screening alerts, deciding whether a flagged name is a genuine match to a watchlisted person or just a coincidence. Part 1 covers the architecture. Before it was allowed to clear a single alert on its own, it had to earn that trust. What earns it is a measured evaluation, run against data the model has never seen and scored on the one error that actually carries a cost.

The asymmetry that decides everything

Every evaluation choice we made follows from a single fact: the two ways the model can be wrong do not cost the same. A false positive is an escalation that should have been a clear, and it costs an analyst a few minutes to confirm there is nothing there. A false negative is a clear that should have been an escalation, and it means a sanctioned person is onboarded, the alert is closed, and no one revisits it.

One error costs a few minutes of analyst time. The other is a regulatory breach that surfaces later, in an audit or an investigation. Because these costs sit so far apart, overall accuracy is the wrong thing to optimize, since accuracy treats both errors as equally bad. We optimized for something narrower and more honest: how rarely the model misses a true match.

The golden dataset, built double-blind

None of this can be measured without a trustworthy source of truth, and the obvious source is not trustworthy. If you score the model against the decisions analysts recorded in production, you are measuring agreement with a label that is itself noisy, because analysts occasionally misread a record or select the wrong option under time pressure.

To get a clean benchmark, we built a golden dataset through a double-blind review. We took real screening cases and had analysts re-adjudicate them without seeing the original production decision and without seeing ScreenGPT’s output. Each case was judged on its own merits, and the label recorded the correct outcome rather than whatever had been clicked the first time. Where the original decision had been wrong, the golden label said so. That is the difference between a benchmark and a transcript.

We also curated the set deliberately. We favored a wide range of scenarios over a large number of similar ones, and we included the difficult cases on purpose: transposed names, missing middle names, absent nationality. Easy cases do not distinguish a good model from a bad one. We balanced the set across analysts, watchlists, and countries, so that it could not quietly mislead us about any single class. The final benchmark held 145 genuine clears and 124 genuine matches.

The scoreboard

ScreenGPT against the double-blind analyst labels, 269 cases. The eight false negatives in the lower left are the only errors that carry regulatory cost. The fifty false positives cost analyst minutes.

We scored ScreenGPT against all 269 cases. The model correctly cleared 95 of the genuine clears and correctly escalated 116 of the genuine matches. It produced 50 false positives, the inexpensive error, and 8 false negatives, the expensive one. The eight false negatives are the smallest group on the board. Set the two error counts side by side and the intent is clear: the model makes the cheap mistake roughly six times as often as the expensive one, which is exactly the balance we tuned it toward.

The one number we protect

The full classification report on the 269-case set. The protected figure is highlighted.

Two numbers from that report carry the argument. The first is recall on the POTENTIAL_MATCH class, which is 93.55 percent. Of every genuine match in the set, the model escalated 93.55 percent and missed 6.45 percent, and those eight false negatives are that 6.45 percent. Because this figure is precisely one minus the error we cannot afford, it is the number we protect above all others.

Precision and recall by class. We accept low NO_MATCH recall in order to keep POTENTIAL_MATCH recall above the target line.

The second number is recall on the NO_MATCH class, which sits at 65.52 percent, and it sits there by choice. Failing to recognize a clear simply means escalating something harmless, which is the cheap error, so we accept it. The model over-escalates by design, and the 50 false positives are that decision made visible. On a hard, unbalanced set you cannot hold both recalls high at once, so we chose the side we could live with.

The headline number understates the model

Overall accuracy of 78.44 percent is a floor rather than the real figure. When we read the disagreements one by one, a meaningful share were cases where the model was right and the recorded label was wrong. Some involved individuals who were deceased, a fact absent from the sanctions data, which the model correctly escalated by procedure. Others were cases where the analyst’s written note agreed with the model while the recorded click did not. Correcting for these, the true alignment is closer to 85 percent. We continue to report the lower figure, because on a compliance system it is better to understate than to overclaim.

Reading all eight misses

The false negatives mattered enough that we did not summarize them; we reviewed all eight. Two were data-entry errors in the underlying record, where the analyst ultimately agreed the model had been right. One turned on an alias that was missing from the record entirely, which no model could have matched. The remaining five all reduced to a single question: how aggressively should a partial, transposed, or one-letter-different name be treated as a match. The misses were not scattered lapses in reasoning. They collapsed into one fixable ambiguity in the policy, and the fix was a clearer rule in the prompt rather than a larger model.

The recurring loss patterns

The same review, run over the fifty false positives, is what turned the evaluation from a scoreboard into a punch-list. Clustered by cause, the errors were not fifty separate problems:

Two things fall out of this table. First, the largest bucket and the five real false negatives are the same pattern seen from opposite sides: inconsistent handling of partial and transposed names. One prompt rule tightens both columns of the confusion matrix at once. Second, the nationality and document buckets, 22 cases together, are not reasoning failures at all. When a field is missing from the watchlist record, escalating is the correct behavior, and the fix is enriching the input data rather than changing the model. This is what evaluation is for: it does not just grade the system, it tells you which part of the system to fix.

Evaluation is a loop

The golden dataset drives the scoring; the confusion matrix and the disagreement review tell you what to fix; the revised prompt is scored again on the same held-out set.

This is the shape of the work. Evaluation was not a gate we passed once. It was the loop that turned a promising model into one we were willing to run unattended against a backlog. What that looked like at scale, and what it cost, is Part 3.

The takeaway

The lesson generalizes well beyond screening. When the cost of errors is asymmetric, you do not optimize for accuracy. You decide which error you can afford, and you protect the metric that guards against the one you cannot. For us that metric was recall on a true match, and 93.55 percent was the number we refused to trade away.