Evidence-Based Policy: What is Missing?

Most of the conversations I’ve been in about evidence-based policy in human services have started from an implicit assumption, roughly as follows: we have lots of good evidence now and the big challenge is figuring out how to get people to use it when they make important decisions. The primary response has been to make evidence more accessible, notably through the creation of directories of evidence-based programs. There’s also a fair amount of research about how to deliver evidence to policymakers (unsurprisingly, “briefly” and “from trusted sources” are high on the list of findings). These are important and useful developments.

Take a long step back, though, and there are some critical elements missing from this picture.

First, it leaves out the decision-makers. They are leaders, not passive consumers of someone else’s knowledge. They make choices about where to focus their attention as they pursue their goals. What evidence do they want, and for what purposes?

Second, it leaves out the decisions. What kinds of decisions do these leaders have to make, and what kinds of evidence are relevant to these decisions? Without good answers to these questions, we have only a fraction of the information we need to promote evidence-based policy. We know a fair amount about how decision-makers like their evidence cooked, but too little about what they want to eat.

The Evidence Decision-Makers Want examines evidence from the perspective of leaders in two very different fields, child welfare and youth employment. These leaders describe a wide range of decisions ranging from selecting strategic priorities to the actions needed to implement those strategies. Decisions about formal programs like those included in the directories make up a relatively small part of their work. They use multiple sources of evidence in addition to published research. Two especially important forms of evidence are analysis of administrative data and the evidence generated by interactions with their customers and front-line staff, usually through interviews or focus groups. Both help leaders understand the needs and experiences of the people they are trying to help.

There’s lots more in the report, for example about the kinds of evidence decision-makers want but have a hard time getting, and about the factors that constrain their use of evidence. And the report concludes with a set of recommendations on how to generate more of the evidence decision-makers want and how to help them use it well. I encourage you to read it and would welcome your feedback.