Your essay is about voice, values, and validity. What is the relationship between voice and validity when we’re talking about an equitable vision for society?
When I think of voice, I think of it in the context of validity—I don’t separate the two. In fact, the reason our understanding of truth is limited is because we treat voice as one thing and validity as another when they are actually closely connected: Multiple voices—multiplicity of voices—increases validity.
Patriarchy, racism, and capitalism have made it more challenging and dangerous for those who don’t have power to speak and to challenge validity; that’s how power keeps power. We see it in the highest levels of government—whenever there are voices that push against the perceived validity of power, they are removed. That’s what dictators do; they remove people that dissent. In this moment, White supremacy, capitalism, and patriarchy are desperately hanging on to power. Because the reality is, there are more of us (e.g., females, non-binary, people of color, Indigenous) than of them, and if they give up their power, they will not get it back.
Does that mean alternative voices should push harder or push differently?
“Those of us who really want a different world for those who follow us have to leverage every asset we have smartly, to push against power maintaining in power.”
It’s never either/or, it’s both/and. People who have less to lose need to push harder; they can push in ways that others cannot. And for those of us who really want a different world we must leverage every asset we have to push against the current power staying in power. We have to get much smarter.
How much of that work should be centered around making room for other voices and other narratives to hold the table?
Again, both/and. At times you have to be the conduit for other voices because it’s safer for you, and at other times you need to move out of the way—sometimes entirely, and sometimes move aside or even right behind so as to have the backs of people whose voices have traditionally been unheard, unwelcomed, or not valued.
Have we set up the rules of the game to allow that?
We haven’t. The table’s just all messed up. Part of it is, who’s setting the table? Why is it a table to which you have to be invited, and not a picnic outside where we as humans are part of the natural ecosystem and there is more space for folks to join? Even our metaphors say something about what we value. They suggest ownership of a house that has a table—one of reasonable size.
As it is now, there’s a host of ways we say that people are welcomed, but who decides who is or is not welcome still reflects a power dynamic that privileges existing power holders who are often those who are part of the White dominant culture or have figured out how to operate within that culture.
Some people think of validity as truth. You seem to be saying that there are multiple truths, which creates ambiguity, and that determining whose truth matters most, and in what situation, can be a matter of voice. Is there no objective truth or validity underneath it all?
“The notion that there’s a singular definition of ‘valid’ rejects the complexity of the world we live in.”
The notion that there’s a singular definition of “valid” rejects the complexity of the world we live in. In fact, most data lacks validity because it doesn’t reflect complexity and multiple truths. Now, if you’re doing “science,” randomized controlled trials, there’s some empirically valid data. But that isn’t social science, human experimentation, foster care, childcare, education, homelessness—those things don’t lend themselves to a singular definition of “valid.” In those fields, all validity is contextualized. The idea that we would take this construct developed in the 1960s for a very small array of scientific experiments and think that that frame makes sense for the myriad ways in which we are engaging in human and social experimentation seems ridiculous to me. It was never designed to do that.
Is there a framing of the word or construct “evidence “ that does make sense to you, or is it problematic in and of itself?
As with validity, I don’t believe there is a singular form of evidence. I think it depends. We need to do the work of defining what type of evidence we think is most relevant to the question at hand, and explain the values underlying our assumptions about the interventions/efforts in which we’re engaged and what we expect to see as a result. If we were to make that thinking transparent, we could do a better job of knowing whether we think the evidence is valid enough. But most of us don’t think about it; we just use a definition of validity or evidence that comes from some other time and some other work. It’s that lack of intellectual interrogation, honesty, and transparency that for me is problematic.
In your essay you talk about complexity, and in this conversation, you’ve talked about ambiguity. Say more about how you view those two qualities.
A little context: My head and heart are in a constant state of weaving in new ideas and learnings. As I age, I like to believe I am learning and changing; also, I continue to be drawn to the in-between. So, if you ask me this question three months from now there probably will be a new piece to it. And if I were to write that essay—which was drafted in winter 2019—again, it would also have a different feel. What I offer here is point-in-time thinking.
Let’s start with ambiguity, which presumes a lack of certainty and control. The uncertainty caused by Covid-19 was a new thing for some people, but others have lived with it their whole lives. Who the f*** was ever certain? Only White folks are certain; Black and Brown folks and other racial and ethnic groups are never certain. This whole idea of not knowing what’s going to happen tomorrow—I never knew what was going to happen tomorrow, nor did I expect to know. I am very comfortable with ambiguity; I do not need to know the answer to everything. Many of my White friends, however, are not. I can recount numerous instances when some trivial question will come up and it will take about 15 seconds before they go to Google to find out the answer. There’s something about this need to know, this perception of having certainty and control, that I think is very much part of Whiteness.
“Complexity is about understanding that there are multiple factors in play, and things are emergent, and as one thread is pulled another thread may be pulled as well, and what surfaces is yet to be known.”
That’s different from complexity. Complexity, super simplistically, is about understanding that there are multiple factors in play, that things are not linear, things are emergent, and as one thread is pulled another thread may be pulled as well. It’s about understanding there is a relationship between and among, and what surfaces is yet to be known.
What are the implications for voice, values, and validity in light of societal changes that occurred in 2020 because of the COVID-19 pandemic and the renewed movement for racial justice?
Let me rephrase that question. For me, what comes up is the heightened awareness that our shared ignorance around our collective history has contributed to much of the state in which we find ourselves at this moment. Because our generation of knowledge perpetuates a White-male-dominant frame that oppresses, represses, and gives falsehoods, we don’t understand who we are and our roles. As a consequence, we are victims of a power structure whose only purpose is to perpetuate its own power.
When I say “we” I mean this federated union of states, otherwise referred to as the United States of America: our collective ignorance. We are not cognizant of our histories. Our desire to create a singular story means that one truth gets elevated over another. As a consequence, we don’t understand nuance or complexity or culture. We go for the highest or the lowest meaning, when most of us are in the middle. Our inability to walk that macro/micro line makes us very easy to manipulate, as a whole, and the conservatives have done a masterful job of that. The Covid-19 and systematic racism moment elevated this situation for some, but the reality was always there; it was always going to happen. There are people who have always worn masks, who have always lined up for food, who have never had 24 rolls of toilet paper at one time.
We have all been complicit in some ways, particularly those of us who have the privilege of having intellectual conversations, of falling victim and beneficiaries to norms and preferences that limit and demean the humanity of many others. We perpetuate it. Research does it, evaluation does it, those of us who are part of generating knowledge are creating a particular type of knowledge that privileges and prioritizes a world view that dehumanizes much of our population.
“We have all been complicit in some ways…of falling victim and beneficiaries to norms and preferences that limit and demean the humanity of others.”
What comes next remains to be seen, in all honesty. Jumping to what will be next is part of our desire to make sense of being in dis-ease. I think the desire to go back to “normal” is a complete rejection of the possibility that that norm is gone forever. There’s a strong pull for power to maintain power; that is what it does. “Normal” means that the pecking order remains in check, and I don’t think we in the U.S. have come to grips with the reality that that can’t ever happen, it won’t happen again.
Now, what’s likely to happen, if those in the middle (which as my mother says is anyone who needs their paycheck regardless of its size) don’t pay attention, is that the disparities and inequities will just be greater than before, because power will maintain its power unless we unseat it. The 1 percent will still be 1 percent when we get a handle on Covid-19. There will just be more of us who are poor. But the middle has yet to understand where they fit in the pecking order, and I don’t think they will understand it until there is more death, more loss, and more upheaval.
What needs to happen in the fields of social research and evaluation to achieve the changes you call for in your essay? Who needs to do what?
Again, point in time thinking. Let’s start with evaluators, and let’s break it up generationally: over and under age 40. For the most part, evaluators who are under 40 have been allowed to live into the intersectionality of their identities in ways those of us who are older could not or chose not to. Their ability to embrace complexity and nuance and to see multiplicities of voice and validity is actually better than for those of us who had to pick which of our identities we would lead with and which we would hide in order to survive in the world that exists in this country. For those of us who are older, to the degree we can unlearn what we believe to be true, we have a responsibility to create space to say, write, and do things, to challenge things, that allow those who are coming up to be more of their true selves in their work. That is what I try to do: give cover to younger folks to push on things. The onus to do that is on us.
Then there’s the academy—one of the bastions of White-male-dominant framing. All the research tells us that to get your PhD, and to survive, and to be a whole human being, if you don’t look a certain way moving through the academy—unless you want to be in a “special” discipline outside the mainstream—it’s more than a notion. I don’t think the academy wants to change, let’s be clear about that. If people within the academy want it to change, those within it have to organize and move forward with a shared agenda.
With practitioners, we have to somehow bridge the unhelpful divide between strategy, evaluation, and learning. The ways in which those disciplines have evolved has created some false differences that limit our ability to be critical thinkers, when really it should all be about inquiry whether you’re talking about research or learning or evaluation or even assessment. If you are engaged in any sort of mission-driven organization, your inquiry should be designed to help you understand how you’re making good on your mission in the ways in which your strategy is rolling out. My hope would be if you are a strategist you understand evaluative thinking, and if you are evaluator you understand strategy. Strategy is how you accomplish outcomes, so if I’m your evaluator I need to know what your strategy and your desired outcomes are to help you evaluate. And learning has to be in service of mission.
The non-profit sector—I think there is something insidious about the nonprofit sector (and let’s not even get started on the name) that reflects its historical origins of charity—that “if I do good, that is good enough.” The sector has not totally freed itself from that mindset, so many of its aspects focus on the delivery of services as their purpose for existing, as opposed to the response to the question, “How do we create a world in which my delivery of services is not needed as much?” A tough question when people are fighting for survival, and yet there is something about how the entire system has been set up which prevents us from getting anyplace else than where we are.
The non-profit industry’s ability to accurately think about what it would take for the world to be different, and their role within the ecosystem beyond the delivery of services, is limited. Part of it is generational and part is the nature of the work, but there isn’t a collective conversation going on about what we, as the nonprofit sector, are. Those that come through the other side of this will be in a different marketplace, and it will be interesting to see what happens then.
I have a bone to pick with consumers of knowledge (so, everyone else) as well. I think we as individuals need to get smarter about understanding where data comes from, who paid for that data, and what it does and does not say. Ask questions: Where was it sourced, who do you think they interviewed, is it reliable? Be critical.
“If we don’t consider all the dimensions of knowing to be valid expressions of evidence, we’re only getting a small piece of the pie.”
Philanthropy has a unique opportunity to reimagine and co-create in ways no other sector can do, which is why the Equitable Evaluation Initiative[1] started with foundations. Those that have jumped on the equity bandwagon have to evolve what they believe to be validity and evidence, because otherwise their evaluative practice is often at odds with their programmatic efforts. All the collaboration and inclusion that happens on the program and strategy side goes completely out the window if they use the dominant evaluation paradigm. So, if they’re seeking to be fully aligned, foundations have to make sure their evaluation work is just as holistic and inclusive, and holds just as much multiplicity, as their programmatic work. For others, I would say, “Your data just isn’t as valid as you think it is. If you’re comfortable with that, you should continue to move forward, but it will be limiting”—not just in terms of truth and voices but also in the ways we need to empirically characterize evidence as something that could most likely be quantified. Because if we don’t consider all the dimensions of knowing to be valid expressions of evidence, we’re only getting a small piece of the pie.
[1] Founded and directed by Dean-Coffey