The simplest model that works

There is an obvious move available to a program holding this much data: train a large model, let it find whatever it finds, act on the output. We do not, and the reason is not caution about technology.

Why not use a black-box AI model?

This program exists because a system made consequential decisions about people using an instrument whose logic nobody had to justify — prescription counts, without diagnosis or intent. Replacing an opaque count-based rule with an opaque learned rule reproduces that failure with better mathematics and worse accountability, because at least a count can be inspected.

Who needs to see how the rule works?

1. The patient, who is entitled to know why something happened to them. “The model indicated it” is not an answer a person can do anything with.
2. The clinician, who needs to be able to disagree — and cannot disagree with something that has no stated reasoning.
3. The agency’s analysts, who need to audit what is being applied to their population, including whether it works differently for different groups within it.

What does it mean for an AI model to be interpretable?

Not “we can generate a post-hoc explanation.” A feature-importance chart attached to a black box is a story about the model, not the model. Interpretable here means the rule can be written out in clinical language and argued with:

• A small number of conditions, stated in terms a clinician uses.
• Versioned and dated, so any decision traces to the rule in force at the time.
• Performance reported with its failure modes — including which subgroups it does worse on, which is where automated systems in health care reliably go wrong.
• Compared against the complex alternative, with the comparison reported rather than asserted. If a complex model is substantially better, that fact should be visible, and the decision to use the simpler one should be a decision rather than a default.

And discovery is not decision

The output is a candidate rule for clinical review. Clinicians decide whether it makes sense and whether it should be used at all. A rule can prompt; it cannot act.

One thing we will not build

A per-person suspicion score. Given what the social determinants evidence shows — disability, unemployment, education and incarceration all strongly associated with overdose risk — a model trained to predict misuse from available features would efficiently reproduce existing disadvantage and present it as a finding. The people it flagged would be the people who are already worst off, and the flag would make them worse off still.

More: AI integrated rule discovery.

Frequently asked questions

What is the difference between interpretable AI and explainable AI?

Explainable AI usually means a post-hoc explanation attached to a black box, such as a feature-importance chart. That is a story about the model, not the model. Interpretable AI, as IntellaRx uses the term, means the rule itself can be written out in clinical language: a small number of conditions a clinician would use, which a patient, a clinician or an analyst can read and argue with.

Why does interpretable AI matter in health care?

Because three parties need to see the rule. The patient is entitled to know why something happened to them. The clinician needs to be able to disagree, which is impossible without stated reasoning. The agency’s analysts need to audit what is applied to their population, including whether it works differently for different groups. A rule nobody can read fails all three.

Is a simple rule less accurate than a complex AI model?

IntellaRx does not assume the answer. Every candidate rule is compared against the complex alternative, and the comparison is reported rather than asserted. If a complex model is substantially better, that fact is visible, and choosing the simpler rule becomes a stated decision instead of a default. Performance is reported with its failure modes, including the subgroups where the rule does worse.

Can a clinician override an AI rule?

Yes. The output of rule discovery is a candidate rule for clinical review, not an instruction. Clinicians decide whether it makes sense and whether it should be used at all. A rule can prompt; it cannot act. Each rule is versioned and dated, so any decision can be traced back to the rule that was in force at the time.