Conflicts of interest are widely acknowledged as a concern in research ethics (e.g., Steneck, 2007) and in testing, researcher independence is highly valued when evaluating validity (Kane, 2013). In this talk, I report initial findings from a systematic review of high-stakes English proficiency test validation studies, starting with studies reported in five leading language testing journals over the last five years (2016-2021). Areas of focus will include researcher affiliation, funding sources, conflict of interest disclosure, and broad trends in research methodology such as general approach (quantitative, qualitative, mixed), access to official material (tests, scores, operational data), and sampling.

Much has been written about research ethics in Applied Linguistics as well as in other disciplines. With some exceptions, relatively little of the discussion in this area has focused on the ethical decisions journal editors make. Some ethical issues are clear. For example, if plagiarism is detected, an article will not be accepted and in some cases additional action might need to be taken. But in many instances, the decisions made by journal editors fall into a gray area and may not even be recognized as pertaining to ethics at all. In this presentation, our intent is to problematize some personal experiences we have had as editors and to discuss what we perceived the ethical issue to be and how we resolved those issues. We do this from the perspective of having served as Editor of Studies in Second Language Acquisition (Susan Gass, 2015-2022) and Associate Editor/Senior Associate Editor (Luke Plonsky, 2015-2022; Editor 2022- ).

Questionable Research Practices (QRPs) are often viewed as the “murky waters” of research ethics and cover practices that fall between ideal behavior and absolute misconduct (e.g., falsification). Whether intentional or not, researchers make choices that may lead them to engage in QRPs; for example, with limited time to edit a manuscript, a researcher might opt to leave out some detailed information in the methods section instead of shaving off words in the discussion. This talk reports on a recent survey aiming to investigate (i) what types of QRPs and related ethical considerations researchers in the quantitative humanities in Sweden and the U.S. have encountered and (ii) the perceived severity of these QRPs.

Training materials for research ethics often include case scenarios as they allow scholars to consider ethical dilemmas within a safe space. These scenarios rarely include solutions, instead they rely on the reader to evaluate the evidence in the case and learn the intended lesson.

In this interactive session, the audience will take part in developing solutions for a set of case studies involving QRPs. With permission, materials generated from this presentation will be used as part of an online training toolkit for researchers in the quantitative humanities to provide improved self-study and in-class training on QRPs.