The measure this program is not built to move

It is worth starting with the one that everybody reaches for. Reduction in opioid prescribing volume is not an outcome measure for this program. It is the measure that twenty years of policy optimized, and step four is what happened: prescribing fell while opioid-involved mortality rose 95% to its 2022 peak. A program evaluated on prescribing volume will be steered into producing exactly the harms described in step seven, and will look successful while doing it.

If an agency requires prescribing reduction as the headline measure, that is a legitimate choice and this is probably the wrong program.

What we think should be measured

Harm

• Fatal and non-fatal overdose in the enrolled population, and in the jurisdiction.
• Emergency department presentations related to pain, withdrawal or overdose.
• Mental health crisis events — the tapering literature makes this a required measure, not an optional one.
• Deaths from any cause in the enrolled population.

Continuity of care — the measure that is almost never collected

• Proportion of patients who lose their prescribing clinician during the program.
• Therapy gaps: episodes where a patient went without prescribed medication, and their duration.
• Proportion of patients who leave care entirely. A cohort study found tapering associated with subsequent termination of care at an adjusted odds ratio of 4.3.
• Time to a new clinician for patients who lose one.

Whether the program is actually happening

• Proportion of enrolled patients receiving each component.
• Behavioral health screening completion, and what proportion of critical findings received a clinical response, and how quickly.
• Naloxone in the home, and safe storage in place.
• Delivery reliability — deliveries made on schedule as a proportion of those due.

Population-level

• Time from a change in the local drug supply to a warning being issued.
• Overdose clusters detected, and whether response was reallocated.

How it should be analyzed

By the agency’s own epidemiologists, or by an independent evaluator, with access to the underlying data. Not by us, and not only reported by us. A pilot evaluated solely by its vendor tells nobody anything, and every agency in this field has a shelf of those already.

Measures should be pre-registered where the agency has a mechanism for it, and the analysis plan agreed before the data exists.

Questions this raises

Why not measure reduction in opioid prescribing?

Because that is the metric the last two decades optimized, with the results set out in step four. A program steered by prescribing volume will produce the access harms described in step seven and will report success while doing it. See what a prescribing limit actually does.

Who should analyze the results?

The agency’s own epidemiologists or an independent evaluator, with access to the underlying data. We will supply data and analysis; we should not be the only party producing the findings. See evaluating a pilot honestly.