Lesson 12
Robin Champieux and Heather Coates
Synopsis
Research data have become valued as a first-class research product as advances in computing technology have dramatically shifted the way in which data are generated, managed, shared, disseminated, and reused. Internet-based technologies have revolutionized the sharing and dissemination of research data in ways similar to journal publishing, resulting in a tremendous amount of measurable online activity. This chapter will introduce the basic concepts of research impact metrics, describe how such metrics for data can be used both by scholars and those tasked with evaluating them, and offer practical guidance for disseminating research data in a way that supports visibility, reuse, and the creation of rich metrics. Throughout the chapter, the focus will be on the role, perspective, and actions that individual scholars can take to disseminate and share their data in ways that enable the use of metrics and qualitative evidence for career advancement.
Core concepts & keywords
Research Impact: The demonstrable contribution that research makes to society and the economy. Research impact can be both academic impact (e.g. advancing method, theory and application across and within disciplines) and economic or societal impact (e.g. use in society and the economy to the benefit of individuals, organizations, or nations). (from the Economic and Social Research Council at UK Research and Innovation).
DORA: The Declaration on Research Assessment, argues against the evaluation of scholarly work based on where it was published.
Scientometrics: The quantitative study of scholarly literature, developed metrics used to evaluate scholars and their publications such as the h-index.
Altmetrics: Metrics based on how people interact with scholarly work using online tools and systems, e.g. news mentions, blogs, social media mentions, or download counts.
Citation-Based Metrics: Metrics based on citations, such as the Relative Citation Ratio and the h-index.
Publication Metrics: Metrics used for journal articles, books, and book chapters.
Journal Impact Factor: Publication metric that measures the annual mean number of citations of articles published in a particular journal, not representative of most individual articles.
Data Discoverability: How easy it is for potential users to find out that a given dataset exists; affected by the data depositor's choice in repository.
Activities
Exercises - Practice what you've learned
Make a list of five prominent journals in your field. What is the Journal Impact Factor for each? Do they provide altmetrics, e.g. an altmetric attention score?
Implement these practices in your career
Review Champieux and Coates' list of "Best Practices for Scholars" (see section 7). Consider which of these suggestions you already follow, and which you want to start practicing. Make yourself a plan.
Reread the section "Own your scholarly profile" (see section 3.1.1). Think about the intended audience for your own scholarly profile. How can you make your profile accessible and discoverable?
Calculate your h-index by creating a profile on Google Scholar. Once completed, you will be able to see how many times your publications have been cited by documents on Google Scholar and your h-index. (For more instructions on finding your h-index, see: https://researchguides.uic.edu/c.php?g=252299&p=1683205)
Look at one of your publications or that of a colleague. Try using the Altmetric bookmarklet to see its altmetrics. Do you have any non-citation evidence of impact, such as a news mention, social media discussion, or high usage from another country? How would you communicate the impact of this work in one sentence in a grant proposal?
Quiz - Test yourself!
Relevant data management use cases
Managing data in the development of the Phonetics Information Base and Lexicon database (PHOIBLE) by Steven Moran
Managing treebank data with the Infrastructure for the Exploration of Syntax and Semantics (INESS) by Victoria Rosén and Koenraad De Smedt
Managing speech perception datasets by Anne Cutler, Mirjam Ernestus, Natasha Warner, Andrea Weber
Share your thoughts on this article or topic
Use #LingData #ResearchImpact #Metrics on your favorite social media platform!
About the authors:
Robin Champieux is the Director of Digital Scholarship and Research Engagement at OHSU, where she leads the Library's scholarly communication and research data services. Her work and research is focused on enabling the creation, reproducibility, accessibility, and impact of digital scientific materials. She is the the co-founder of the Metrics Toolkit and Awesome Libraries.
Heather L. Coates is the Digital Scholarship and Data Management Librarian at the IUPUI University Library Center for Digital Scholarship and the Indiana University Data Steward for Research Data. Her work in the library centers on supporting faculty success in research and career advancement. As an open research advocate, she cares deeply about the integrity, accessibility, and sustainability of the scholarly record as a public good. She is a co-founder of the Metrics Toolkit.
Citations
Cite this chapter:
Champieux, Robin and Heather L. Coates. 2022. Metrics for evaluating the impact of data sets. In The Open Handbook of Linguistic Data Management, edited by Andrea L. Berez-Kroeker, Bradley McDonnell, Eve Koller, and Lauren B. Collister, 157-170. doi.org/10.7551/mitpress/12200.003.0016. Cambridge, MA: MIT Press Open.
Cite this online lesson:
Gabber, Shirley, Danielle Yarbrough, Andrea L. Berez-Kroeker, Bradley McDonnell, Eve Koller, Lauren B. Collister, Robin Champieux, and Heather L. Coates. 2022. "Lesson 12." Linguistic Data Management: Online companion course to The Open Handbook of Linguistic Data Management. Website: https://sites.google.com/hawaii.edu/linguisticdatamanagement/course-lessons/12-metrics-for-evaluating-the-impact-of-data-sets [Date accessed].