The failure mode to avoid

The standard pilot in this field enrolls a small, cooperative, easily reached group, runs for a few months, reports process measures, and produces a favorable narrative that nobody can act on. It is not dishonest; it is just uninformative, and it is uninformative in a predictable way.

What makes a pilot informative

1. A population defined by a rule, not by convenience. Everyone in a defined geography meeting defined criteria, including the people who are hard to reach — because they are the population the program claims to help and the one it is most likely to fail.
2. A comparison of some kind. A matched population, a stepped-wedge rollout across sites, or at minimum a well-characterized pre-period. Without one, every finding is confounded with time, and time is doing a great deal in this field right now: opioid-involved deaths nationally fell from 82,851 in 2022 to 55,007 in 2024 with no help from anybody’s pilot.
3. Long enough to see the outcome you care about. Overdose is a low-frequency event. A three-month pilot in a small population cannot detect a change in it, and reporting one would be noise.
4. Pre-registered measures, including the negative ones from measurement and evaluation.
5. A stopping rule. Agreed in advance: what result would cause this to be halted early. A program with no stopping rule cannot fail, which means it cannot succeed either.
6. An independent evaluator with access to the data.

On publication

We think pilot results should be published regardless of what they show, and we will agree to that in writing. The literature in this field is distorted by the pilots that were never written up, and a foundation running a program of this kind has no good reason to add to that.