a work in progress by Dr. Vidhya Shanker, PhD (last modified 6/24/22)

Repair

Evaluation's Canon

Suggested citation: Shanker, V. Why is Evaluation So White? Accessed at https://sites.google.com/view/whyisevaluationsowhite on [INSERT TODAY’S DATE].

The Invitation

Your Host

Vidhya Shanker, interdependent consultant and co-convener of the Minnesota IBPOC in Evaluation Community of Praxis. A big finding of my dissertation is that several critical voices representing indigenous communities, black communities, and communities of color have been suppressed in evaluation in favor of vague discourses of diversity, culture, and inclusion. As part of my dissertation research, I had a chance to interview some of the bearers of these critical voices, but the nature of my research question did not necessarily allow for full exploration of their story on their own terms. Ever since, I have had the dream to assemble a posse to collectively resurrect that history—of IBPOC and otherized elders more generally who have been erased by the canon—by conducting oral histories with them. I think it would be instructive to learn more about what led them on their path, what got them through/ kept them going, and where they would like to see us and the field go now.

You

You want to restore balance by amplifying the critical voice of otherized evaluators and stabilizing dominant, liberal narratives of difference, disparities, and disproportionalities. Note that white women who are considered authorities in diversity, cultural competence, and inclusion are not what this effort is about.

The Frame

The purpose of this invitation to repair the canon is to document the labor, contributions, and insights of marginalized voices to evaluation—specifically to its theory and practice regarding racialized difference and systemic oppression more generally—in hopes that subsequent generations of IBPOC and otherized groups engaged in movement organizing know that they have a lineage in evaluation and do not necessarily have to figure everything out all by themselves. This invitation is rooted not in diversity, equity, cultural competence, or inclusion. It is rooted in abolitionist, liberatory, and emancipatory conceptualizations of intersectional, epistemic humility and justice. It is in the spirit of Nobody Knows My Name, through which Stafford Hood and Rodney Hopson have begun documenting the lives of African American evaluators and connecting their contributions to larger social issues. Pamela Frazier Anderson and Tamara Bertrand Jones have begun this exploration with African American women evaluators.

The Offering

I have a list of approximately 20 evaluation scholars and practitioners who the document analysis piece of my dissertation suggest have actively contributed to the field’s understanding and practice regarding racialized difference and systemic oppression. I also have a list of approximately 80 who were more tangentially involved. These lists are limited to the USA and—of course—do not include the numerous folks whose contributions even within the USA were never documented anywhere (that would be part of the ask, below). But I can offer these lists as a point of departure.

I can also offer a compilation of the oral histories that the field of evaluation has completed through its Oral History Project. This Project currently consists of 16 oral histories—of exclusively white and nearly exclusively men evaluators—that have been printed in the American Journal of Evaluation. We could consider riffing off these for a protocol that we collectively develop. We could deliberate on whether we want our oral histories incorporated into the Oral History Project or not—I have been in touch with them and could include what I learned within what I have to offer.

Additionally, I have some acquaintances who may be able to train us in qualitative methodologies that I could offer introductions to.

Finally, I have some historical context and perspective gained from my dissertation research that I could offer.

The Ask

I am asking folks to join me in honoring the labor and contributions of indigenous folks, black folks, folks of color, queer/ trans folks, and folks with disabilities to the field through oral histories of them. This may involve:

  • identifying and soliciting funding to underwrite expenses involved in doing the field of evaluation this service;

  • reading work by and about some scholars in the above lists, as a starting point;

  • identifying folks whose thinking has been evaluative/ role has been pivotal, but for whom there is no existing documentary evidence;

  • thinking critically through our sampling and methodology;

  • bringing in other related experience, including with oral history writing specifically;

  • providing, orchestrating, or engaging in great out-of-the-box trainings on qualitative methodologies with us;

  • reading the existing (white) oral histories for ideas about protocol and to understand and interweave contemporaneous histories;

  • co-creating appropriate protocol(s);

  • contacting identified scholars/ practitioners;

  • collecting photographs and other documentary evidence/ artifacts;

  • conducting the oral histories themselves;

  • transcribing them/ getting them transcribed or potentially editing video and audio recordings;

  • preparing final manuscripts, recordings, digital exhibitions;

  • identifying potential vehicles for publication and other types of dissemination (e.g., AEA365, etc.).

We may consider (virtual?) group interviews, wherein folks of the same—and different—eras could engage in conversation, reminiscing, and storytelling with each other.

The Outcome

I envision publications and other creative products coming out of this collective process. I am very interested in the possibility of journal articles culminating in a book and interactive exhibition (a physical installation as well as an audio, video, and/or virtual experience), perhaps at an upcoming AEA and/ or CREA conference in addition to other venues.

The larger outcome is the cultivation of intergenerational bonds among otherized evaluators and illustration to members of incoming generations, who often feel isolated, that we have always been evaluating—informally as well as formally—and that we have a lineage and a place in this field of evaluation.

The Acceptance