Questions worth asking any vendor in this field

If you are an agency evaluating anything in this space, these are the questions that separate a program from a pitch. We have tried to answer all of them somewhere on this site; if we have not answered one, that is worth knowing too.

On evidence

1. “Which of your components has demonstrated effectiveness, and which does not?” A vendor who cannot name a weak component either has not looked or is not telling you. Ours are on one page.
2. “Show me the primary source for that figure.” Not a slide, not a white paper — the paper or the government dataset. Then check that the figure is actually in it and that the population matches.
3. “What population was that studied in?” The most common way a true figure becomes misleading.
4. “What would make you say this had not worked?” If there is no answer, there is no evaluation.

On legal exposure

1. “Does HIPAA apply to us for this?” — and be wary of anyone who answers. That is a legal determination about your agency and it belongs to your counsel. A vendor answering it is either guessing or assuming liability they cannot carry.
2. “Does any of this touch substance use disorder information?” If yes, ask separately about 42 C.F.R. Part 2, because it is a distinct regime and being outside one thing does not mean being outside everything.
3. “Give me the data inventory.” Every element, its source, its purpose, its recipients, its retention. If that document does not exist, the answer to every governance question is improvised.

On what the thing actually does

1. “What does a participant see, and what can they refuse?” If refusing has a consequence for their care, it is not consent.
2. “What does my agency see about an individual?” The answer should be very little.
3. “Who else gets this data?” Ask about subcontractors, laboratories, and any analytics partner.
4. “What happens to the data if we stop?”
5. “Can this be used to support an enforcement action?” Ask it directly and get the answer in the contract.

On the measurement

1. “Who analyzes the results?” If the answer is the vendor, alone, the results are marketing.
2. “Is there a comparison group?” National overdose deaths fell by about a third between 2022 and 2024. Any uncontrolled program running in that window will report improvement.
3. “Will the results be published if they are bad?”

More: how engagement works · pilot design.