Why Compliance Scores Must Be Reproducible

Here is a question every compliance buyer should ask a screening vendor: "Run the same subject twice, a year apart. Will you give me the same score?" If the honest answer is "probably, if the model hasn't been retrained" — walk away.

A score is not just a number; it is a claim about evidence. When you clear a case at 23 or escalate one at 78, you are asserting that a specific body of facts supports a specific conclusion. If the machinery that produced that number can drift, the claim drifts with it — silently, and long after the decision is filed.

What reproducibility buys you

1. Examiners can re-run your logic. In every mature AML jurisdiction, the examiner's playbook includes asking how a score was derived. If your vendor's model is stochastic — or worse, an opaque API endpoint that nobody can pin to a version — the derivation is a mystery even to you.
2. Decisions stay explainable forever. A case cleared in March must be re-defensible in December. With versioned, deterministic scoring, the trace records the engine version, and the score can be recomputed from the original inputs byte-for-byte.
3. You can actually test your control. Regression suites, threshold tuning, list updates — all of it is meaningful only if the baseline is stable. Nondeterministic scoring makes "tuning" a euphemism for gambling.

Deterministic doesn't mean dumb

There is a myth that explainable, deterministic systems are somehow less capable than stochastic ones. For name matching and rules-based typologies, the opposite is true: the state of the art is deterministic. Jaro–Winkler similarity, token-set matching, factor-weighted scoring — these are pure functions. Feed them the same input, get the same output, on any machine, in any year.

The Mizan engine is a pure function with an explicit version tag:

// Same inputs → same score. No sampling, no drift, no surprises.
score = f(subject, watchlist_record, policy, engine_version)

// engine v1.0:  name_similarity 0.94  · DOB exact +8
//              distinctiveness +4  · jurisdiction +22
//              composite → 87 CRITICAL

Every trace line carries the engine version. When we ship v1.1 with a better transliteration table, the audit log knows which version produced which decision — and the old version can be re-run against new list updates for re-screening, without re-deciding history.

One more consequence worth naming: deterministic scoring runs identically on-premises and in the cloud, with zero data egress. Institutions under the strictest data-residency requirements get the same engine, the same scores, the same traces — because there is nothing about the engine that depends on where it runs.

Ask your vendor the question at the top of this post. If they hesitate, you have your answer. If they say "of course", ask to see the version tag on the trace. We'll show you ours.