Core Area 2: Teaching, Learning, and/or Assessment Processes

2a: An understanding of teaching, learning and/or assessment processes

Description

I am responsible for the deployment of a numerical, quantitative peer assessment in a number of our Psychology courses. I have worked with four academics over the course of my time at the University of Edinburgh to discuss, design, implement and review a procedure that has had a notable effect on student participation in their courses. It was imperative during this process that I kept a strong awareness of assessment policy in addition to ensuring that the assessment fit the academics’ teaching preferences.

The process began in 2022 and there have been two iterations of the assessment: once in the first half of 2023 and the second in the first half of 2024. The assessment has changed considerably between the two iterations, due to changes in requirements as well as student and staff feedback. Involvement in this project has necessitated consistent process review, as well as regular communication with the academics involved to ensure it still meets requirements.

For the first iteration of the assessment, the academic had very specific requirements which meant the options I had were inflexible when taking into account the software available.

The requirements were as follows:

·            Students would undertake an assessment (a lab report) in groups

·            They would submit the group assessment to Learn

·            Then they score their colleagues out of five on their participation

·            If a student received an average of >3, this would give them 10% extra credit for this assessment

·            If a student received an average of <3, the 10% extra credit would not be added and they would just receive the group score

After proposing some ideas and discussing with the academic, I finalized the following process using Learn Original Course View:

·            Learn Group Assignments were set up for the report, where one student could submit on behalf of a whole group

·            WebPA assignments were used for the quantitative peer review element

·            The scores were then extracted to an Excel spreadsheet

·            Formulas were used to identify which students should receive extra credit and the final score for each student

·            These scores were then inputted into Learn

Reflection

After one iteration of this procedure, it was clear to me that something needed to change as it required a great deal of manual input from the academic and myself, and there was room for human error. As this was affecting students’ grades, I stressed the importance of avoiding any possibility for error and streamlining the process as much as possible. After this iteration and its success, other academics were keen to implement a similar procedure, so I wanted to make sure that the process met the needs of the course but also was sustainable and future-proofed. Although the first iteration was very successful and both staff and students were content with the way it ran, I believed it could be made more efficient while maintaining its efficacy.

I sat down and discussed the requirements with the academic and expressed that I could alter the assessment, making the most of WebPA as a tool as it is meant to be used, rather than involving excess manual input for us both. I proposed that:

·            As before, Learn Group Assignments (this time in Learn Ultra) were used for the report submission

·            WebPA assignments were used as before, making the most of the in-built formula to fairly grade students based on the scores given to them by their colleagues

·            Linked via LTI, scores are then transferred from WebPA to Learn and students can view their final scores

I created a visual for the academic to show them how the assessment scores would be worked out using the formula.

Evidence

Once communicating this, I confirmed the procedure and the assessment went ahead for the second time. This time, there was far less manual input and the process was smoother. Feedback from Dr Umberto Noè is as follows:

“A key component of the course involves completing practical data analysis tasks in the labs.
  Abby has helped instructors move from individual work to group work. She created group sets on LEARN, which allowed staff to manage a large number of groups in a smooth way. She also helped us by implementing peer assessment of contribution using the WebPA tool. The setup was easy and straightforward to use for instructors, tutors, and students.
  Before the introduction of group work with peer assessment of contribution using WebPA, student attendance was roughly around 20%. After the introduction of peer assessment of contribution using WebPA, attendance was roughly 80%. Students were more likely to attend if they knew that their contribution affected their grade.”

I am pleased with the way this assessment now runs as students are participating more in their group assignments, increasing the significance of the assessments and encouraging active learning. I am glad that I reviewed and proposed a new version of this process, as it has meant I am able to support more academics in adding a peer element to their course. As with all new processes, communication with the course organiser was imperative, as well as regular check-ins as I learnt a lot from him about the ins and outs of assessment in his subject area. I was also grateful to speak to Learning Technology colleagues across the college to share best practice and get advice from those who have used the tools.

Three more academics now undertake the same process in their Psychology courses, and although I support and advise in the creation and maintenance of these assignments, after training and communicating the procedures, they feel empowered enough to manage them themselves. As word spreads, the sustainability and future-proofing of these assessments becomes more important, so that less effort is spent on manual input and more effort is spent supporting staff and students in their teaching and learning processes. In addition, as our VLE evolves, there may be scope for a less complicated set-up in the future, which will open up possibilities across the faculty.

2b: An understanding of your target learners

Description

In my position as an eLearning Advisor at the University of Aberdeen, I was involved in the design, delivery and support of a five-week course called Designing and Tutoring Online Courses, directed at staff members and PhD students with teaching responsibilities. In March 2020, due to the pandemic and the move to online learning the – previously blended – course moved to fully online and the number of students rose from roughly ten per cohort to between sixty and a hundred.

As the nature of the learners changed, the teaching and assessment procedures had to change to meet new requirements. The five-week option was maintained but a five-day intensive version was also introduced, which condensed the material and made it more relevant to the general staff body. The types of learners that signed up for the five-day course was different to that of the five-week version, as the former attracted all staff with teaching responsibilities, while the latter had more depth and attracted those who were interested in creating online and blended courses. This needed to be taken into account when deciding which learning technologies should be deployed for each course.

The aim for the course differed depending on the duration. For the five-day course, the aim was to showcase the tools on offer and empower staff to make use of them in their now-online courses. For the five-week course, alongside the aims of the five-day version, the goal was to give the participants an experience akin to that of an online or blended student. By the end of it, they would see what set up they found engaging or educational, and would have a deep understanding of what they wanted to incorporate in their own versions of the courses.

One of the elements across the two versions of the course was a problem-based-learning (PBL) activity, where previously participants would participate over the course of five weeks (both online and in person) to find a solution to a complex course-related problem. For the five-day version, this was translated into a two-hour synchronous online activity through the use of Blackboard Collaborate, breakout groups and online resources, followed by an optional asynchronous discussion board.

Evidence

Reflection

Something I had to take into account was the nature of the learners, who in March 2020 may have had some trepidation around using web conferencing software. I also wanted to ensure that the tools were being showcased fully, so the participants could understand the benefit of using them and may be inclined to follow suit in their own courses. Although in this case they were the students, they would later be using the information from the course as tutors, so creating an engaging and educational exercise was necessary. After each cohort, I participated in a feedback session, where comments would influence the structure of future variations. I also sent out a Microsoft Forms which asked prompting questions to figure out how to ensure we were making use of the time allotted in the most efficient way. 

The synchronous, online exercise went as follows:

·            Students joined an online BB Collaborate session

·            I would introduce the session and explain the activity

·            Students would be put into breakout groups (5/6 per group)

·            Within the groups, they would assign the roles of tutor and scribe to two of the group

·            They would be given a scenario to solve, the tutor would direct and the scribe would take notes

·            They would then return to the main room and the scribe would report back on behalf of each group

What I found useful about this exercise was the organisation of it: roles were decided before the participants reported back so there was no ambiguity around this. I wanted to make sure that there was as much clarity as possible as, during the pandemic, people were stressed and – at this point – not used to working from home. This also allowed for those uncomfortable with presenting online due to reasons such as neurodiversity, introversion and anxiety to take an equally important but less stressful role. The introductory part of the session was imperative for this, and I also made sure to consistently drop into breakout groups so the participants would have the chance to ask questions outside of the main room. As time went on, I was glad that word spread about the course and popularity grew. I ended up hosting back-to-back iterations of the course which were consistently fully subscribed.

If I were to undertake this sort of exercise now, I would perhaps incorporate a flipped learning element, where students could read the scenarios ahead of time, so that time in the sessions could be used for more in-depth discussion. I would add a formative personal journal into the exercise, where participants could consider scenarios ahead of the synchronous session. I understand that often students, particularly those studying part time, are less likely to complete non-compulsory elements so I would encourage participating by incorporating the content from the journals into the structure of the synchronous sessions. I believe this could allow the time in the session to be spent more efficiently and allow more discussion time post-breakout room. I would also consider incorporating the PBL as part of a wider, feed-forward assessment process, where the result of the activity would feed into the assignment which followed and a student could undertake individual research based on this. Alternatively, the scenarios could be altered so they are relevant to the rest of the assessments and the knowledge gained would be useful for the future elements of the course. I would like to incorporate this kind of assessment in my current role and I am working on an accessibility course currently which could make use of some of these techniques.