Evaluating Scholarship in the Humanities: An Exploration & its Impact on Tenure, Promotion, and Progress
“In evaluation, publication is universally understood as the chief objective evidence of achievement”
Guillory, 2005, p. 32
Introduction
Research and scholarship in the Humanities spans over various disciplines such as philosophy, literature, languages, women’s and gender studies, political theory, history, jurisprudence, and the arts (The Heart of the Matter, 2013 as cited by Lui, 2014). Research is conducted in various ways, however, typically involves engaging with the existing literature and discourse, examining where a gap in the knowledge exists, and formulating research questions that can be answered either qualitatively or quantitatively through the collection of data. The dissemination of scholarship from the Humanities typically looks like the publication of a book in addition to journal articles, but this is changing. Oftentimes scholars from the Humanities disciplines will engage with colleagues and peers throughout their research process through conferences, online forums, and the like, but will usually work independently for the completion of their work (Stanford Humanities Center, 2017; Thelwell & Delgado, 2015, p. 824). However, collaboration for projects does occur regularly.
Evaluation of scholarly work in the Humanities has been a topic of major discussion, not only for measuring impact in its respective subfield, but also in how to grant scholars tenure and promotion in departments at universities. Discourse and debate regarding evaluation of scholarly work within the Humanities tends to center on the various and numerous nuances of research that is produced in this discipline and how to properly evaluate said research; in other words, the literature suggests that there is a lack consensus about how to measure and assess Humanities research (Guillory, 2005; Daniel &Van den Akker, 2016). Another component that further complicates this ongoing conversation is the notion that questions the validity and accuracy of relying solely on citation impact for evaluative purposes (Mohammadi & Thelwall, 2014; Thelwall & Delgado, 2015; Daniel &Van den Akker, 2016).
In the following report, these two factors will be briefly examined for the purposes of understanding the intricate dynamics at play specifically when it comes to tenure and promotion processes and procedures in higher education. In addition, this brief discussion will hopefully shed light on current practices, how the horizon is changing, and what a future of evaluating Humanities scholarship could potentially incorporate. The importance of this topic rests on how scholarship in the academic landscape is both changing and slowly witnessing the inclusion of different kinds of scholars into the conversations taking place allowing for the incorporation of diverse ideas, questions, and research methods. With the evolving landscapes in research and scholarly communication, comes the transformation of the direction of dialogues taking place. As such, the ways in which research is assessed necessitates an equal and continuous transformation of evaluative practices in academia.
Tenure and Promotion Practices
The tenure process in the United States is just that, a huge and lengthy process that is based on a scholar’s production of work and contribution to the production of knowledge. Rebecca Clay, for the American Psychological Association (2018), summarized what the tenure process generally looks like. To be considered for tenure, an academic needs to be hired in a tenure track position. Tenure review will typically start at year six. Once this year comes up, a scholar will usually submit a portfolio like record of accomplishments such as “publications, grant funding, list of courses taught, sample syllabi, student evaluations and a summary of…service to…department, institution and field”; additionally, a scholar will “also submit a list of possible external evaluators” while the department will submit a list of their potential evaluators (Clay, 2018). Generally, decision making for actually being granted tenure is contingent upon a twofold process where in tenured faculty in a scholar’s department provides their decision and then an institutional wide committee—and sometimes, board of trustees—will also provide a review and decision (Clay, 2018).
There are of course more details involved in such a serious process; there are hearings, letters, evaluations, secrecy, and votes (Guillory, 2005). Universities and colleges will often times provide their own guidelines and principles for granting tenure; and overall, it is difficult to obtain such a promotion, especially if you are a woman or a person of color (Strunk, 2020; Bradley et al., 2017). There are some scholars who feel this process is archaic, exclusionary, and in need of reform (Deresiewicz et al., 2021; Flaherty, 2021). For instance, Kamden Strunk (2020), an Associate Professor of Educational Research at Auburn University, believes that the American tenure and promotion process perpetuates the status quo where scholars of color and women face additional systemic and institutional challenges in being granted tenure. He also believes that the tenure process lacks an ethic of care and rather dehumanizes candidates into a removed mechanism in an established and deeply judgmental operation that views scholars as undeserving (Strunk, 2020).
As previously mentioned, publications are a major component regarding being granted tenure. And in the Humanities, published work will largely, though not exclusively, take the form of a monograph. Journal articles are also a common practice; however, promotions usually are contingent in tenure track positions with publishing a book (Guillory, 2005).
Evaluative Practices of Scholarly Work
In regards to publications, evaluating published scholarly works takes on many forms especially when it comes to contribution to a researcher’s field. There are citation metrics and analysis, downloads as well as journal impact factor, various index measures, and peer review (Mohammadi & Thelwall, 2014; Pendlebury, 2009). These measures, specifically dealing with citations, “generally…represent the notions of use, reception, utility, influence significance, and… ‘impact’ (Pendlebury, 2009, p. 6). As previously mentioned, publications along with contribution to one’s field is a consideration when a scholar is up for tenure at an academic institution. Pendlebury (2009) indicates that these tools to evaluate, however, do no take into account quality of work—that is a matter that is largely difficult to measure without the presence of human subjectivity (p. 6).
In the natural and physical sciences, producing and publishing may arguably be quicker—though just as rigorous and stressful process—due to the distribution of knowledge and publications through internet mediums and results in more rapid disciplinary consensus (Guillory, 2005, p. 32). On the other hand, Guillory (2005) points out, that in the Humanities, producing knowledge looks and feels a little different because it takes longer due to monographs being the standard; as such projects are usually slower and long-term (p. 36). Additionally, because of the ongoing crisis in scholarly publishing, this presents yet another challenge in Humanities publications because of the scarcity in resources and overall lack of demand on a community level but simultaneous push and pressure to publish on a departmental/institutional/academic level (Guillory, 2005; Future of Scholarly Publishing, 2003).
Influence and contribution to an academic discipline brings into question the idea of success in the sciences an in scholarship. Bornmann and Marx (2012) examine this idea of success in scholarly research where they utilize the Anna Karenina Principle as a starting framework from which to view, understand, and think about success in scholarly endeavors in a context [academia] where resources, such as funding, space in a journal, recognition, and reception are scarce. They do assert, however, that this principle in determining and evaluating success is useful primarily for the sciences. They believe that further research is needed for this to be applied (Bornmann & Marx, 2012, p. 2048); in regards to the Humanities, social sciences, and qualitative research endeavors, additional research and inquiry is imperative.
Another component further complicating an already intricate matter, is the pressure researchers face to publish as much as possible to render them competitive and worthy candidates in the tenure review process. Scholars are pushed to produce and contribute to the discourse as quickly as possible and get their names out in their fields. Guillory (2005) and the Modern Language Association (2003) touch upon this too and discuss how speed and quantity dominate the field. This undoubtedly can and likely has impacted the quality of publications. Guillory (2005) states, “As a system-wide tendency, raising standards by demanding more publication is paradoxically likely to result in a decline in the quality of scholarship and a creeping cynicism about publication” (p. 33).
Evaluating in the Humanities
As explained in the previous section above, there are a lot of components to evaluating scholarly research that situates the work in the Humanities in a precarious and difficult position. Some questions that come up are: how do committees adequately evaluate scholarly work in the Humanities? How can established processes in evaluation properly provide repeatable methods? Are these practices equitable? How do digital Humanities come into play? Are the established measures such as citation metrics and analysis and bibliometrics as useful for evaluations?
The literature indicates that there are no clean or easy answers to those questions, however, conversations and debates are being had, which means that progress is being made. Daniel and Van den Akker (2016) declare that the Humanities needs to become more organized; they believe that if the Humanities does not work to become an organized body, respecting the differences in each subfield, then assessment will always lack consensus, stunting progress in the discipline as a whole (p. 27-28). They continue that much of the challenges arise because of all that makes up the Humanities; it is a discipline that is large, extensive, and dense in subject matter and there’s a lot that needs to be considered (p. 25).
Furthermore, the ways in which work in the other sciences has been evaluated, cannot easily be translated into the Humanities simply because of the way that this discipline operates. For one, “…there is not currently enough evidence to be reasonably sure that citation count data would help to improve the accuracy of peer review judgements about arts and Humanities research because it would be much weaker as an indicator of scholarly impact” (Thelwell & Delgado, 2015, p. 826). So, while citation impact, citation analysis and metrics can glean some information for evaluation in the Humanities, it does not provide a comprehensive picture, including for tenure and promotion purposes. Guillory (2005) believes that “It is of course a desirable outcome for scholarly work, but I suggest that evaluation in the context of tenure should not make the fact of publication the only objective form of scholarly accomplishment” (p. 32).
Mohammadi and Thelwall (2014) believe that altmetrics and usage and readership data can also be a remedy, or a supplement, to aid in Humanities evaluation of scholarly work because it brings more of that which was formerly not visible within these measures such as impact through social media, blogs, and online reference managers (p. 1628).
The digital world is also changing the landscape when it comes to assessing and evaluating impacting tenure and promotion as well. Books and other work can now be directly published as eBooks, online, or digitized allowing for a wider audience to access the work and therefore impact altmetrics, and usage and readership data. Guillory (2005) believes that there is an extensive and salient reliance on scholarly publication as a measure of achievement and success
(p. 36). Perhaps the digital world and the expansion of the Digital Humanities has the potential to also transform the heavy dependency on traditional ways of disseminating and presenting work.
Conclusion
Tenure, promotion, scholarly publishing, and the production of knowledge all form part of the conglomerate that is academia and research. Well established practices within the ivory tower have historically kept women, scholars of colors, and individuals with diverse identities and backgrounds on the margins within the space of higher education. With that being said, scholarly work in the Humanities, as an evaluative measure for tenure, is on a trajectory that guarantees eventual transformation because change is already taking place. This evolution into progress incorporates varied methods from which to evaluate and distinct products that go beyond a monograph or journal articles.
Hopefully this concise discussion offers a digestible glimpse at the expansiveness that is assessment in the Humanities. The current and overall recent and established methods for evaluating scholarly work and specifically work in the Humanities underscores how complicated, nuanced, and intricate this topic truly is. As the path of scholarly work evolves, so will the research that attempts to explain it and render it accessible to larger audiences.
References and Works Cited
Bornmann, L., & Marx, W. (2012). The Anna Karenina principle: A way of thinking about success in science. Journal of the American Society for Information Science and Technology, 63(10), 2037–2051. https://doi.org/10.1002/asi.22661
Bradley, D., Yerichuk, D., Dolloff, L.-A., Galway, K., Robinson, K., Stark, J., & Gould, E. (2017). Examining Equity in Tenure Processes at Higher Education Music Programs. College Music Symposium, 57. Retrieved from https://www.jstor.org/stable/10.2307/26574458.
Clay, R. A. (2018). The tenure process. Monitor on Psychology, 49(7). Retrieved December 5, 2021, from https://www.apa.org/monitor/2018/07-08/tenure-process.
Daniel, H.-D., & Van den Akker, W. (2016). Yes we should; Research Assessment in the Humanities. In M. Ochsner & S. E. Hug (Eds.), Research assessment in the humanities towards criteria and procedures (pp. 23–29). essay, Springer International Publishing.
Deresiewicz, W., White, D. E., Beilock, S., Afinogenov, G., Ocampo, A. C., Garofalo, D., Daut, M. L., Thelin, J. R., Helfand, D. J., Hamlin, K. A., Miles, C., Stein, L. A., Wolinetz, C., Zaloom, C., Mangum, T., & Thorp, H. (2021, November 18). The future of tenure. The Chronicle of Higher Education. Retrieved December 5, 2021, from https://www.chronicle.com/article/the-future-of-tenure?bc_nonce=dnr725k9xbqtn6eebm6yz8&cid=reg_wall_signup.
Flaherty, C. (2021, May 14). The DEI Pathway to Promotion. Inside Higher Ed. Retrieved December 4, 2021, from https://www.insidehighered.com/news/2021/05/14/iupui-creates-path-promotion-and-tenure-based-dei-work.
The Future of Scholarly Publishing: MLA Ad Hoc Committee on the future of Scholarly Publishing. (2003). Journal of Scholarly Publishing, 34(2), 65–82. https://doi.org/10.3138/jsp.34.2.65
Guillory, J. (2005). Valuing the Humanities, Evaluating Scholarship. Profession, 28–38. Retrieved from https://www.jstor.org/stable/25595795.
How is humanities research conducted? Stanford Humanities Center. (2017, June 12). Retrieved December 6, 2021, from https://shc.stanford.edu/how-humanities-research-conducted.
Liu, A. (Ed.). (2014, December 21). What Are the Humanities? 4humanities: Advocating for the Humanities. Retrieved December 6, 2021, from https://4humanities.org/2014/12/what-are-the-humanities/.
Mohammadi, E., & Thelwall, M. (2014). Mendeley readership altmetrics for the Social Sciences and Humanities: Research Evaluation and Knowledge Flows. Journal of the Association for Information Science and Technology, 65(8), 1627–1638. https://doi.org/10.1002/asi.23071
Pendlebury, D. A. (2009). The use and misuse of journal metrics and other Citation Indicators. Archivum Immunologiae Et Therapiae Experimentalis, 57(1), 1–11. https://doi.org/10.1007/s00005-009-0008-y
Strunk, K. K. (2020, March 13). Demystifying and Democratizing Tenure and Promotion . Inside Higher Ed. Retrieved December 9, 2021, from https://www.insidehighered.com/advice/2020/03/13/tenure-and-promotion-process-must-be-revised-especially-historically-marginalized.
Thelwall, M., & Delgado, M. M. (2015). Arts and humanities research evaluation: No metrics please, just data. Journal of Documentation, 71(4), 817–833. https://doi.org/10.1108/jd-02-2015-0028
Presentation Link: