The simplest model that works

There is an obvious move available to a program holding this much data: train a large model, let it find whatever it finds, act on the output. We do not, and the reason is not caution about technology.

The reason

This program exists because a system made consequential decisions about people using an instrument whose logic nobody had to justify — prescription counts, without diagnosis or intent. Replacing an opaque count-based rule with an opaque learned rule reproduces that failure with better mathematics and worse accountability, because at least a count can be inspected.

Three parties who need to be able to see the rule

1. The patient, who is entitled to know why something happened to them. “The model indicated it” is not an answer a person can do anything with.
2. The clinician, who needs to be able to disagree — and cannot disagree with something that has no stated reasoning.
3. The agency’s analysts, who need to audit what is being applied to their population, including whether it works differently for different groups within it.

What interpretable has to mean

Not “we can generate a post-hoc explanation.” A feature-importance chart attached to a black box is a story about the model, not the model. Interpretable here means the rule can be written out in clinical language and argued with:

• A small number of conditions, stated in terms a clinician uses.
• Versioned and dated, so any decision traces to the rule in force at the time.
• Performance reported with its failure modes — including which subgroups it does worse on, which is where automated systems in health care reliably go wrong.
• Compared against the complex alternative, with the comparison reported rather than asserted. If a complex model is substantially better, that fact should be visible, and the decision to use the simpler one should be a decision rather than a default.

And discovery is not decision

The output is a candidate rule for clinical review. Clinicians decide whether it makes sense and whether it should be used at all. A rule can prompt; it cannot act.

One thing we will not build

A per-person suspicion score. Given what the social determinants evidence shows — disability, unemployment, education and incarceration all strongly associated with overdose risk — a model trained to predict misuse from available features would efficiently reproduce existing disadvantage and present it as a finding. The people it flagged would be the people who are already worst off, and the flag would make them worse off still.

More: AI integrated rule discovery.