Why is evaluation so white?


Useful resources to explore:

“I have been in too many meetings where a racialised person has felt they’ve had to speak about their lived experience, at great personal cost […]. Sometimes, the individual’s point is directly challenged or downplayed. In a head-spinning moment of gaslighting, they are left isolated and disbelieved, despite (or, perhaps, because) they are the racialised person specifically invited to the meeting to explain why the racist thing is racist.”

‘I’m sometimes asked, “Why are there so few people of color in evaluation?” I flip the question: “Why is evaluation so white?” And answer: “Because our labor is actively erased.”

‘Of the 35 recipients of the Paul F. Lazarsfeld Evaluation Theory Award since 1977, 28 evaluators listed in the sacred Evaluation Theory Tree published in 2004, 22 evaluators featured in the related Evaluation Roots book published in 2004, and 16 evaluators featured by AEA’s Oral History Project since 2003, not one has been a woman of color or indigenous woman.

‘This omission can lead us to conclude that for the last 40 years, no women of color or indigenous women have been academically trained as evaluators, conducting formal evaluations, and engaged in scholarship on evaluation—let alone engaged in evaluative thinking and critical inquiry that are considered outside the boundaries of evaluation. Or that their work is fringe.

‘The evaluation work of several women of color and indigenous women allows us to do the work that we do every day. This post aims to repair the miseducation of evaluators of all ages and experience levels…’

Further resources, e.g.,

It’s striking how issues discussed in the 70s are still relevant now, e.g., concerning the impossibility of using IQ tests (and covert proxies thereof) to improve outcomes rather than simply to blame a child and excuse education systems for poor outcomes.

‘At its core, evaluation is value laden and embued with and responsive to a larger social political order, and evaluators are situated within contexts of study and within interactions of the setting that shape the evaluation study’s logic, structure, and practices (Hopson, Greene, Bledsoe, Villegas, & Brown, 2007). The question of “Who evaluates and why?” highlights the contexts, agendas, and intentions of the evaluation and the evaluator and so raises questions about practices—sometimes commonly accepted ones—and the structures of power and the uses of those power structures for or against hegemony.’ [p. 418]

This cites The oral history of evaluation, part 3: The professional evolution of Michael Scriven, which provides a clue – hiding in plain sight – to why the official history of evaluation is as it is (bold emphasis added):

“Now, there was the May 12th group, which was ahead of the game. The May 12th group was so called after the first date on which they met [in “about 1968”, says Gene Glass–AF]—but the general feeling was if we call it the May 12th group, that will have absolutely zero cachet, and so no one will be able to argue that they were entitled to join the May 12th group because it’s called something generic. And so the idea was you got invited to the May 12th group, and if you weren’t invited, then you weren’t in, and so there was no official stuff. So, they would meet in somebody’s house once a year. […] But some of us felt that we needed to do something that was slightly more official, and we’d got to start making this more than the intellectual elite group.”

The May 13 Group formed on that date in 2020 to challenge this.

‘Evaluation is political. At its simplest, evaluation is the systematic “process of determining the merit, worth and value of things” (Scriven, 1991, p. 1). Who gets to decide, the questions, the process, and the criteria for determining merit, worth, value, or significance—all of these matter.’ [p. 534]

‘As professionals and practitioners, we can no longer sit on the sidelines wearing the cape of objectivity and neutrality, a cape that shields beliefs and assumptions about knowledge, rigor, and evidence and which elevate a Western White worldview. [..] Everyday narratives that continue to marginalize, minimize, and disrespect people of color and those with less privilege could be replaced with ones that do not demonize and place blame on the individual. They could instead lift up the historical, contextual, and powerful dynamics that create and sustain oppression and shed light on the strategies and solutions which can shift the “rules of the game,” so that equity is achievable.’ [p. 538]

“Advisors of evaluation graduate students of colour should create spaces for students to express their feelings and, if they choose, be vulnerable and open about the stressors of simply being a person of colour in a world with white supremacy woven into its very fabric. “

“Whenever a prospective student emails me, I put them in touch with current students in my department. I find this is especially important for international students; I am unable to speak to how the culture in North Carolina and in our department differs from their home culture. I also aim to introduce students to faculty across campus who have similar cultures and backgrounds”

“Advisors of evaluation graduate students of colour can research or have conversations about the norms and dates associated with the holidays and events that their students observe. […] While I can’t know all the traditions observed by my students, I encourage them to inform me about their cultural and religious traditions as appropriate.”

“… advisors and mentors should also practice giving microvalidations […], small acts and words that validate who graduate students believe they can be. My post-doctoral advisor always praised me in public and raised concerns in private. I regularly let my advisees know that I am proud of them, see their potential, and believe in them. I learn every student’s name and work to pronounce their names correctly. And I make a concerted effort to refer to my advisees as my colleagues.”

“… evaluators of color noted that the burden of addressing DEI and calling out racism is often placed on them as they are assumed to be experts…”

“… evaluators of color cited examples of being tapped to join an evaluation project when philanthropic clients asked for demographics of staff in their RFPs, yet not feeling meaningfully included in the subsequent work…”

“When organizations have difficulty retaining staff of color, they often perceive the person of color as the problem, not the ecosystem that reinforces inequities. Persistent challenges with retention should signal a need for the organization to self-reflect on its culture and make changes…”