EPHCS - Module 1

Module 1: A research perspective

Module 1 covers what we know, from a research perspective, about minimum English proficiency levels for healthcare professionals. The first part briefly explains what standard setting is, and the second part reviews the relevant research and points out gaps where more research is needed.

1.1 Introduction to standard setting

This section explains what standard setting is, why it is done, and how it works.

1.1.1 What is standard setting?

Standard setting is the process used to distinguish between test takers who have achieved a certain level of competence in whatever the standard setters are interested in, and test takers who have not achieved that standard. This is accomplished by determining cut-score(s) on the test that is/are associated with the minimum level of knowledge or skill required for that level of competence. In other words, a cut-score is a score or mark that distinguishes between competent and incompetent test takers.

Typically, for certification purposes, standard setting only requires setting one cut-score to determine ‘pass’ or ‘fail’. For instance, through standard setting, it is determined that for Test A, a minimum score of XX is required for test takers to show competency in Skill ZZZ.

Tests may sometimes distinguish between more than one level of competence. For instance, Test B is similar to Test A, but distinguishes between test takers who are minimally competent in Skill ZZZ (they may need some supervision when performing tasks requiring this skill) and those who are very competent in Skill ZZZ (they can independently perform tasks requiring this skill). Standard setting for Test B would require setting two cut-scores: a minimum score of XX shows test takers are minimally competent and a minimum score of YY shows test takers are very competent.

1.1.2 Why is standard setting important?

Standard setting is required to set appropriate cut-scores. Especially for high-stakes tests where outcomes have major consequences for the test taker, cut-scores, which are used for classifying test takers, must be set appropriately, or else there would be doubts about the outcome of the test.

1.1.3 How does standard setting work?

While there are multiple standard setting methods, they are usually conducted in a panel/workshop style, which combines expert judgment and technical considerations, such as difficulty level of tasks from test taker data. A group of carefully selected experts convene and

are trained in the standard setting process and familiarize themselves with the test and test tasks,
before going through multiple rounds of reviewing and discussions on test tasks and/or performances,
which ultimately results in determination of the cut-score(s).

The next section reviews the current standard setting research on English proficiency levels in healthcare.

1.2 Current research

There is no doubt that language proficiency plays an important role in communication. This is especially important in healthcare settings, where miscommunication can have dire consequences, hence the need to set minimum proficiency levels for those working in this area. It is also important to acknowledge that other factors, such as profession-specific language skills (e.g., bedside manner), that are often not covered in language proficiency tests are also involved in communication.

There is limited research on English proficiency levels in relation to healthcare, with most of it being on minimum proficiency levels for registration. In this section, we present some of these findings.

1.2.1 Minimum proficiency levels for registration

The standard setting literature on minimum proficiency levels for registration for health professions mainly come from the US and are for nursing, as can be seen in the table below. Very little is known about the proficiency levels required in other settings and for other health professions. In an ideal world, multiple standard setting studies would be conducted for each setting (e.g., country), for each setting, and for each language proficiency test.

References

Berry, V., O'Sullivan, B., & Rugea, S. (2013). Identifying the appropriate IELTS score levels for IMG applicants to the GMC register. . Report submitted to the General Medical Council. Centre for Language Assessment Research (CLARe), The University of Roehampton, London

O'Neill, T. R. (2004). The minimum proficiency in English for entry-level nurses: TOEFL. NCLEX Psychometric Research Brief, 1. Retrieved from http://www.ncsbn.org/pdfs/TOEFL_Research_Brief_vol_1.pdf

O'Neill, T. R., Buckendahl, C. W., Plake, B. S., & Taylor, L. (2007). Recommending a nursing-specific passing standard for the IELTS examination. Language Assessment Quarterly, 4(4), 295-317.

O'Neill, T. R., Marcs, C., & Wendt, A. (2005). Recommending a minimum English proficiency standard for entry-level nursing. JONA's Healthcare Law, Ethics, and Regulation, 7(2), 56-58.

Qian, H., Woo, A., & Banerjee, J. (2014). Setting an English language proficiency passing standard for entry-level nursing practice using the Michigan English Language Assessment Bettery. NCLEX Technical Brief, retrieved at https://www.ncsbn.org/14_NCLEX_technicalbrief_SettinganEnglishLanguageProficiencyPassing.pdf

Woo, A., Dickison, P., & de Jong, J. (2010). Setting an English language proficiency passing standard for entry-level nursing practice using the Pearson Test of English Academic. Retrieved from NCLEX Technical Brief. National Council of State Boards of Nursing. Chicago, IL

1.2.2 Minimum proficiency levels for prospective health students

To our knowledge, there are no standard setting studies on the minimum English proficiency levels for entry into health programs. Therefore, from the existing literature, one can only work backwards from entry levels for registration and estimated English proficiency gains from the health program. Although many people expect that students would show reasonably sizeable gains during the course of study in terms of improved performance on tests of English proficiency, the literature on the language development of students over their entire university degree (not health degrees in particular) shows quite small gains for those starting at higher proficiency levels (i.e., around IELTS 6 or 6.5), which correspond to the proficiency levels usually expected for entry into health programs.

Naturally, other factors affect gains during study, including the amount of language support available to and accessed by the students, as well as students’ contact with English outside their programs of study (and outside the university). In addition, it is likely that students improve their discipline-specific language skills during their studies and this would not be reflected in general English proficiency tests. Research would need to be conducted to verify if this is indeed true.

In sum, more research is required to empirically support minimum English proficiency levels for entry into health programs. If these levels are indeed lower than those required for minimum registration requirements, it is expected that they would/should not be much lower.

References

Humphreys, P. (2016). English language proficiency in higher education: Student conceptualisations and outcomes. Unpublished doctoral dissertation. Griffith University, Queensland, Australia.

Knoch, U., Rouhshad, A., Oon, S. P., & Storch, N. (2015). What happens to ESL students' writing after three years of study at an English medium university? Journal of Second Language Writing, 28, 39-52.

O’Loughlin, K., & Arkoudis, S. (2009). Investigating IELTS exit score gains in higher education. IELTS Research Reports, 10, 95-180.

The next module focuses on the four most common English proficiency tests accepted for entry into healthcare programs.

Module 2

Page updated

Report abuse