Alternative approaches to final written exams

(Utviklingsprosjekt, UNIPED-100 Universitetspedagogisk Basiskompetanse 2020/2021)

Abstract

The final exam is undoubtedly a source of stress and anxiety for students and teachers. Despite the by now common knowledge of the influence of assessments on students’ learning strategy, the traditional final written exam is the typical form of assessment in many courses. However, the exam questions often test no more than memorization and repetition; design of exam questions that truly test understanding and transfer of knowledge is challenging. On the other hand, grading exams is often a mind-blowingly boring and frustrating activity for the teachers. In this project, I discuss the evolution of my assessment style from final exam only to multiple assessments and present an attempt at making the final exam more meaningful and interesting for everybody involved. The paper is based on experiences from two courses, one on bachelor level at UiT and one on master level at the University Centre of the Westfjords in Iceland, which follow roughly the same course outline and content but in different settings.

Introduction

Assessment serves different purposes in higher education (e.g. MacLellan, 2001; Newstead 2003; Fletcher et al., 2012, and references therein). It greatly influences student learning by impacting motivation and approaches to learning (the so-called backwash; e.g. Entwistle, 1996; Boud, 2000; Ramsden, 2003; Struyven et al., 2005, and reference therein; Biggs & Tang, 2011; and many more). It provides feedback to the students and teachers about the achieved learning, and information to determine student grades (e.g. Ramsden, 2003, Biggs & Tang, 2011). Traditionally, a large component (if not the only) of course assessment is the final exam, written or oral, despite the artificial setting of such a task (Villarroel et al., 2020; Muldoon, 2012; Biggs & Tang, 2011).

As a student, I did not experience final exams as particularly stressful. However, as lecturer, I find them the by far least enjoyable part of teaching. Especially the grading – in particular going through each answer, reading one exam paper after the other, spending hours on (mostly) little inspiring writing – is a painful exercise that I dread every time. I therefore decided to use this assignment to improve my understanding of the underlying theory to assessments and exams, and to try out a different setup for the final exam. In this paper, I briefly introduce some basic theory, followed by a description of and reflection on the assessment in my courses. I then present the latest change in the exam setup in one of my courses and evaluate its results using student feedback and own reflection. The paper concludes with an outlook to future adjustments and questions raised during the research to this project.

Final exams in the context of assessment and learning

In general terms, assessment falls into two categories (Biggs & Tang, 2011): Formative assessment is used to give feedback to students (and teachers) about the achieved learning, which then can be used to improve the learning (and teaching). Ideally, students should feel free to take risks in their learning and make errors, in order to get useful feedback, learn how to use such feedback, but also learn to assess themselves on their work. Summative assessment is used to determine a grade for the students’ performance or achieved learning. Different types of assessment can be used for both formative and summative purposes, and on occasions, the boundaries between the two can become blurry. However, it should be clear whether an assessment is used for formative or summative purposes in order to let the students decide how to prepare and approach the assessment; combining summative and formative assessments in one might counteract the learning effect of feedback (Knight, 2002; Hernández, 2012).

The final exam (written or oral) is the archetype of a summative assessment. This is problematic in a number of ways:

• High stakes exams might not fulfill the role they had been given in higher education as they fail to assess the range of skill and competencies universities are expected to equip their students with (Knight, 2002). Additionally, they contribute to heightened anxiety, which in turn impacts learning and performance (Røykenes et al., 2014; Trifoni & Shahini, 2011).

• Student perception of the final exam can lead to shallow approaches and prevent deep learning; depending on the exam setup and the students’ interpretation of the task, strategic approaches might allow for deep learning though (Entwistle & Entwistle, 1991; Struyven et al., 2005).

• For increasingly diverse student groups, “traditional” exams and particularly high stakes exams can prove to be efficient barriers first to academia and then to academic success and achievement (Muldoon, 2012; Kwon et al., 2017).

• There is no room for improvement through feedback, and any feedback given after the exam often is not seen as useful or ignored altogether (Carless, 2007; Hernández, 2012).

• Written exams are an artificial setting that does not correspond to the sort of assessment (including peer- and self-assessment) that is required “in real life” (Boud, 2000; Boud & Soler, 2016).

• Despite attempts at making exam grading reliable and objective, e.g. through use of grading rubrics or assessment criteria, there are clear limitations especially for open question essay exams or exams that encourage creativity (e.g. Knight, 2002; Perrella et al., 2015).

To provide both formative and summative assessment in a course, multiple types of assessments should be incorporated. However, assessment methods do not automatically assess a particular skill or competence, or always invoke the same learning approach. Instead, they influence learning dependent on student perceptions, learning environment, motivation, and a multitude of other factors (Struyven et al., 2005). Each assessment method has its own strengths and weaknesses, which lecturers need to be aware of (Challis, 2001; Newstead, 2003; Struyven et al., 2005). In this context, the final exam is just one piece of the assessment puzzle, albeit an important one.

Course setting – general course overview

I teach two courses in introductory oceanography. The first one is an intensive, mandatory, 2.5 week-long course for Master students in Coastal and Marine Management at the University Centre of the Westfjords (UW) in Ísafjörður, Iceland. The student group comprises around 20 international students with very varying backgrounds, i.e. from humanities to marine sciences. Many students are mature students. They are highly motivated, engaged, and often with a clear goal for after graduation. Because of the small community in Ísafjörður, the small size of the university centre (two Master’s programs with 10-20 students each), and the program structure with one intensive course at a time, the students often form a very tight-knit group and have close contact to the lecturers. I took over this course as sole lecturer in 2014, and inherited the previous teacher’s course design and material, which I developed further since then.

The second course is a semester-long, non-mandatory course at the Department of Arctic and Marine Biology at BFE/UiT. It is aimed at 2^nd and 3^rd year biology students. However, it attracts a large number of exchange students each year, with these students accounting for up to 75% of the student group. Therefore, student background and especially literacy in maths and physics varies, though within the natural sciences and not as widely as at UW. Each year, several Master students attend the course. Student motivation varies and can range from “this was the only available course in English” to “I don’t need the ECTS and am only here because I want to learn about the ocean and the climate”. The general course description (including the assessment form) was first submitted and approved in spring 2014, before I started in my current position in summer 2014 and took over as course leader. I then designed the course structure and content and have been teaching the course with support of 1-2 teaching assistants since spring term 2015.

Development of assessment methods in my courses

Apart from spring semester 2015 at UiT, the assessment in my courses consisted of several components that required use of different skill sets to achieve the intended learning outcomes (Figures 1 & 2). In all years (with the exception of UiT in 2015), the final exam was the dominant component though, accounting for at least 40% and up to 70% of the final grade.

My teaching and the assessment design changed over the years as I became more familiar with the literature in education science. However, course and assessment design remain strongly influenced by (my) time constraints and work load considerations, and to a lesser degree administrative constraints.

At UW, the final exam was a written exam from the beginning of my teaching there. At UiT, I moved from an oral exam in 2015 to written exams in the following years. This was very early in my teaching career and before I became more familiar with education science. The decision was therefore based on personal experience and perception of oral exams. Similarly, adjustments in the formulation of questions in the written exams was mostly based on personal reflection on the student answers and to a degree on discussion with the students. Written exams were mostly based on question design following Biggs&Tang (2011)’s ordered-outcome items, attempting to make use of Bloom’s taxonomy and SOLO levels, or open questions, challenging students and providing high achieving students an opportunity to demonstrate their level of learning. However, given the time constraints of an in-class written exam and the introductory course levels, questions tended to remain at a fairly basic level, making the grading of ever-the-same answers simply boring and tedious.

The modified exam: home assignments

In 2020, I changed the final written exam in my UW course to home assignments (done in groups of three, to reduce grading work load). Five assignments were given throughout the course with deadlines a few days after publishing the assignment. These assignments were formulated to include a basic review of underlying oceanographic principles and concepts to a topic, demand independent research to gather information, and apply the basic concepts in order to sort, summarise and present the information found. I did not have the answers to the higher hierarchy questions. These assignments were summative in that they resulted in a grade, but they allowed for feedback along the way and thus improvement from one assignment to the next. Limited guidance was given when asked for. An example of such an assignment in comparison to a tradition written exam question is given in Figure 3.

Evaluation

I received student feedback on the exam through general course evaluation (Appendix A) and a short questionnaire send to them a couple of months after the course (Appendix B). The general feedback was positive. The students engaged with the questions and appreciated the additional time they could use on the assignments. They perceived slightly higher learning. However, there also were clear negatives: The number of assignments was too high, which potentially lead to less engagement and loss of depth in the approach to answering the questions. Two comments mentioned the group work as potentially problematic and reducing the learning effect.

From my perspective, this type of assignment was clearly more enjoyable. The answers varied in their approach and thus in their findings. While the basics were usually covered in similar fashion (though to varying levels of competence, just as would have been expected in a written exam), the more advanced part of the assignment was mostly very good and engaging; the students clearly made an effort to find relevant sources and managed to apply their course knowledge to extract, condense and present also complicated information. I did not specifically ask for a written essay, and one group took the opportunity to create podcasts to two of the assignments. Occasionally observing the students working on the assignment left me with the impression that removing the time pressure of an in-class exam allowed for more engagement with the task, discussion of the subject amongst the entire class, and thus deeper learning.

Since this was the first time I tried this form of assignment, there clearly is room for improvement. As the feedback shows, the number of assignments was too high and needs adjustment; a lower number of assignments could also allow for individual responses instead of group work. The timing of when to publish the assignments needs to be revised; in 2020, this was partly dictated by me being in quarantine during the first week of teaching whereas the second week took place in person.

How to take this further

In spring 2020, the written exam in the UiT course was changed to a home exam due to COVID-19 restrictions. In 2021, there will again be a home exam, however, I decided to increase the time of the exam from three hours to one week. I hope to create similarly inspiring and engaging questions for this course, keeping in mind the different student group, learning environment, and expectations on Bachelor level, and other pitfalls as discussed by Bengtsson (2019).

The research to this project did what most research does: It provided me with more new questions than answers. I failed to find any literature regarding the teachers’ attitude to grading (only a brief hint in Newstead, 2003) – am I the only one to strongly dislike grading written exams? If the teachers’ attitude to teaching, learning and assessments has an impact on students’ learning, what about the teachers’ attitude to grading?

In general, there is a massive amount of literature on the topic of assessment, exams, evaluation etc, and trying to condense this to something useful for the individual teacher is challenging and time consuming. I would appreciate more guidance by UiT regarding UiT’s assessment philosophy, strategy or culture, as this was (and still is?) sorely lacking when I started to teach here.

References

Bengtsson, L. (2019): Take-home exams in higher education: A systematic review. Education Sciences, 9, 267, doi:10.3390/educsci9040267

Biggs, J. & Tang, C. (2011): Teaching for quality learning at university. Open University Press, Maidenhead, England

Boud, D. (2000): Sustainable Assessment: Rethinking assessment for the learning society, Studies in Continuing Education, 22:2, 151-167, doi:10.1080/713695728

Boud, D & Soler, R. (2016): Sustainable assessment revisited. Assessment & Evaluation in Higher Education, 41:3, 400-413, doi:10.1080/02602938.2015.1018133

Carless, D. (2007): Differing perceptions in the feedback process. Studies in Higher Education, 31(2), 219-233

Challis, M. (2001) Portfolios and assessment: meeting the challenge, Medical Teacher, 23(5), 437–440

Entwistle, N. (1996): Recent research on student learning, in: J. Tait & P. Knight (Eds) The Management of Independent Learning, Kogan Page, London, pp. 97–112

Entwistle, N. J., & Entwistle, A. (1991): Contrasting forms of understanding for degree examinations: the student experience and its implications. Higher Education, 22, 205–227.

Fletcher, R. B., Meyer, L. H., Anderson, H., Johnston, P. Rees, M. (2012): Faculty and students conceptions of assessment in higher education. Higher Education, 64:119-133, doi:10.1007/s10734-011-9484-1

Hernández, R. (2012): Does continuous assessment in higher education support student learning? Higher Education, 64:489-502, doi:10.1007/s10734-012-9506-7

Knight, P. T. (2002) Summative Assessment in Higher Education: Practices

in disarray. Studies in Higher Education, 27:3, 275-286, doi:10.1080/03075070220000662

Kwon, S. K., Lee, M., Shin, D. (2017): Educational assessment in the Republic of Korea: lights and shadows of high-stake exam-based education system. Assessment in education : principles, policy & practice, 24 (1), p.60-77

MacLellan, E. (2001): Assessment for Learning: the differing perceptions of tutors and students. Assessment & Evaluation in Higher Education, 26(4), 307-318

Muldoon, R. (2012): Is it time to ditch the traditional university exam? Higher Education Research & Development, 31(2), 263-265

Newstead, S. (2003): The purposes of assessment. Psychology Learning and Teaching, 3(2), 97-101

Perrella, A., Koenig, J. Kwon, H., Nastos, S., Rangachari, P. K. (2015): On being examined: do students and faculty agree? Advances in Physiology Education, 39:320-326, doi:10.1152/advan.00078.2015

Ramsden, P. (2003): Learning to teach in higher education. RoutledgeFalmer, Abindgon, UK.

Røykenes, K., Smith, K., Larsen, T. M. B. (2014): ‘It is the situation that makes it difficult’: Experiences of nursing students faced with a high-stakes drug calculation test. Nurse Education in Practice, 14, 350-356, doi:10.1016/j.nepr.2014.01.004

Struyven, K., Dochy, F., & Janssens, S. (2005): Students’ perceptions about evaluation and assessment in higher education: a review. Assessment & Evaluation in Higher Education, 30:4, 325-341, doi:10.1080/02602930500099102

Trifoni, A., Shahini, M. (2011): How does exam anxiety affect the performance of university students? Mediterranean Journal of Social Sciences, 2(2), 93-100

Villarroel, V., Boud, D., Bloxham, S., Bruna, D., Bruna, C. (2020): Using principles of authentic assessment to redesign written examinations and tests. Innovations in Education and Teaching International, 57:1, 38-49, doi:10.1080/14703297.2018.1564882

Appendix

Appendix A – Exam-related comments from the general course feedback

Question 13: How can we improve the course?

- Only 3 group assignments

- This course had five assignments rather than a final exam because the first week was online. Due to timing issues, these assignments (mini group papers) were not available until the second week of classes. Therefore, we had to complete five test-essays in 10-12 days of classes. This timeline overlapped with our final project and our cheat sheet, only one of which could be finished ahead of time.

I felt that the beginning of the course was too easy and the end of the class was backloaded (likely due to zoom).

- The group work assignment was only focused on Iceland, would have been nice to have some variety there or own input.

Question 14: What are the main short comings of the course?

- The written assignments (group questions) were a good replacement for an exam. However, writing five of them seemed excessive due to the amount of time they took, and detracted from being able to spend time on other work. Fewer of these assignments would be better.

- Group essays were more work than the amount learnt I felt. I think 5 was too many and I think a different assignment or focusing more on one of the other assignments would have been more effective.

Appendix B – Questionnaire answers; scale from 0 = Do not agree to 4 = fully agree