Questions worth asking any vendor in this field

If you are an agency evaluating anything in this space, these are the questions that separate a program from a pitch. We have tried to answer all of them somewhere on this site; if we have not answered one, that is worth knowing too.

What should you ask a vendor about evidence?

1. “Which of your components has demonstrated effectiveness, and which does not?” A vendor who cannot name a weak component either has not looked or is not telling you. Ours are on one page.
2. “Show me the primary source for that figure.” Not a slide, not a white paper — the paper or the government dataset. Then check that the figure is actually in it and that the population matches.
3. “What population was that studied in?” The most common way a true figure becomes misleading.
4. “What would make you say this had not worked?” If there is no answer, there is no evaluation.

What should you ask about HIPAA and legal exposure?

1. “Does HIPAA apply to us for this?” — and be wary of anyone who answers. That is a legal determination about your agency and it belongs to your counsel. A vendor answering it is either guessing or assuming liability they cannot carry.
2. “Does any of this touch substance use disorder information?” If yes, ask separately about 42 C.F.R. Part 2, because it is a distinct regime and being outside one thing does not mean being outside everything.
3. “Give me the data inventory.” Every element, its source, its purpose, its recipients, its retention. If that document does not exist, the answer to every governance question is improvised.

What will the program actually do with people’s data?

1. “What does a participant see, and what can they refuse?” If refusing has a consequence for their care, it is not consent.
2. “What does my agency see about an individual?” The answer should be very little.
3. “Who else gets this data?” Ask about subcontractors, laboratories, and any analytics partner.
4. “What happens to the data if we stop?”
5. “Can this be used to support an enforcement action?” Ask it directly and get the answer in the contract.

How do you know the results are real?

1. “Who analyzes the results?” If the answer is the vendor, alone, the results are marketing.
2. “Is there a comparison group?” National overdose deaths fell by about a third between 2022 and 2024. Any uncontrolled program running in that window will report improvement.
3. “Will the results be published if they are bad?”

More: how engagement works · pilot design.

Frequently asked questions

What questions should you ask an opioid program vendor?

Start with evidence: which components have demonstrated effectiveness and which have not, the primary source for every figure, and the population it was studied in. Then ask what would count as failure, who else receives the data, whether it can support an enforcement action, who analyzes the results, and whether bad results will be published. We would rather you asked us before signing.

Why does a comparison group matter when judging results?

Because the background is moving. National overdose deaths fell by about a third between 2022 and 2024, so any uncontrolled program running in that window will report improvement. A comparison group separates what the program did from what was already happening. And if the vendor alone analyzes the results, the results are marketing.

Should a vendor tell you whether HIPAA applies to your agency?

No. Whether HIPAA applies is a legal determination about your agency, and it belongs to your counsel. A vendor who answers it is either guessing or taking on liability it cannot carry. What a vendor should hand over is a data inventory: every element, its source, its purpose, its recipients and its retention.