This blog is part of CSSP’s six-part #Evidence4Equity series, where we invite evaluators, researchers, and foundation leaders to elaborate further on some of the issues raised by the publication, Placing Equity Concerns at the Center of Knowledge Development, and to share their reflections on how they are intentionally focusing on equity in knowledge development.
Join us on the blog monthly for a new entry diving deeper into these issues and be sure to join the conversation online using #Evidence4Equity to keep the conversation going.
Equitable evaluation isn’t simply about measuring whether outcomes are equitable. It’s also a way to challenge ourselves to consider how our thinking, strategies, and actions may inadvertently perpetuate existing power structures in society, and in philanthropy’s relationships with communities. The opportunity to start improving this dynamic compelled the Learning & Evaluation team at The Colorado Health Foundation (CHF) to move toward a more equitable evaluation practice.
In 2017, CHF underwent a major transition in mission and strategy, and adopted three guiding cornerstones:
- We serve Coloradans who have low income and have historically had less power or privilege.
- We are informed by the community and those we exist to serve.
- We do everything with the intent of creating health equity.
This process clarified the Foundation’s purpose and called us to reflect on the way we approach everything that we do. For the Learning & Evaluation team, the health equity cornerstone became central to our practice. As with any new practice, we struggled with the question of how to start, but challenged ourselves by asking what it would take to put equity at the center of our evaluation work. In 2018, we encountered the Equitable Evaluation Initiative, which helped us begin shifting our thinking, yet we found ourselves saying, “We don’t know much yet. Maybe we should wait until we know more.” We recognized that common paralysis—the idea that we need more training, more time and more knowledge. We worry that we’d make mistakes, but we believed in the value of getting started on practice change, regardless of how ready we felt. Our team began identifying small opportunities to make our evaluation practice more equitable, and over time, we took on bigger changes. Nearly two years later, our learning continues. While we certainly don’t have it all figured out, I can share a few stories from our journey so far.
Recruiting Evaluation Partners
The first thing we changed was our request for qualifications (RFQ) process. This was an obvious starting point because we release RFQs regularly, and they determine our partners and evaluation approaches. We updated the RFQ language to prioritize equity in the design and process of the evaluation. While evaluation partners weren’t required to have expertise in equitable evaluation, we looked for a willingness to explicitly wrestle with issues of how equity should play out in a study.
Crafting Evaluation Studies
Another obvious place to integrate equity was in our evaluation studies–through questions, design, and methods. We started by integrating basic outcome questions about equity in every study, to get at who is best and least well served, and who is left out entirely. We also integrated evaluative questions that assessed equity in the strategy itself, such as:
- What power do those most impacted have on the design and implementation of the work (both ours and that of grantees)?
- Who are we listening to, and whose perspectives are we prioritizing?
- What are we hearing that challenges our mental models about the problem or the solution?
Disrupting Our Own Thinking
Let’s be honest–evaluation prioritizes certain methods and technical expertise at the expense of other ways of measuring or knowing the world. As we came to understand this, we recognized we had some unlearning to do, and partnered with the Center for the Study of Social Policy to re-examine how we thought about evidence. Our intent was to move away from rooting credibility of evidence in methods (e.g., quasi-experimental designs, randomized control trials), and task ourselves and our partners to think critically about what truly underlies reliable, valid evidence. This new thinking allowed us to start having new conversations, both internally and with partners, regarding how we might understand progress and impact. To keep growing in our thinking and practice, our Learning & Evaluation team carves out time monthly to critically reflect on our work. Using various frameworks and articles, we critique and question our current practice, think about how we want to evolve, and hone our awareness of what equitable evaluation might look like in different situations.
It was surprisingly easy to start making our evaluation practice more equitable. We didn’t need to spend months crafting a strategy. We found success by identifying small day-to-day ways we could align how we work and think with principles of equity.
It can feel awkward at times to be learning a new way of doing and being, but placing equity at the heart of our approach is not only a more just way of practicing our craft, but it also helps us do higher-quality evaluation. Starting small, experimenting and paying attention to results, helps us see the next set of changes we need to make. This is a journey, and we intend to steadily keep moving towards more equitable practices.