5.3.2. Recommended strategies in moderation of marking and grading

Methods of moderation of marking must be robust enough to give reasonable assurance of standards, and those should be appropriate to the               assessment component/task. Where different questions in an examination paper/assessment component are set & marked by different First                  Examiners, it is recommended that the moderation takes place at the level of each question to serve the purpose. The faculties should decide,               document and report what moderation strategies it would adopt to achieve the requirements listed in annexure 8.20 in relation to marking and

grading.

➢  At least the strategy A and C shall be considered critical steps for any taught course/module/subject that will contribute to the final GPA or the Class of the degree awarded.

➢     Steps A, C & D, shall be compulsory for courses where evaluation of answer scripts is conducted employing panel marking.

➢     Faculties are encouraged to document their practices on each study programme based on these guidelines.

A. Double-Blind Marking: Minimum standards recommended. 

At least 15%-20% of scripts, and/ a minimum of 10 (& maximum of 50) scripts shall be marked by the Second Examiner (Some universities advise all scripts to be double marked, which is a best practice but considered that it is a step beyond moderation).

  Course of actions in double-blind-marking

i. If the discrepancy between the first and the second marking is less than 15% for any examination component, the marks of the first Examiner shall be accepted. (Universities in some other countries advise the tolerable discrepancy to be <5%.)

ii. If the discrepancy between the first and the second marking is more than 15% in 33% or more of the double marked scripts (some universities does not tolerate this much of a difference [15%] even for a single script), the second Examiner shall mark all scripts and the average mark of first and second examiners shall be used for grading. The Chief Examiner shall decide on this, and the first Examiner shall be informed of this decision.

iii. When these mechanisms have not led to a resolution, the External Moderator shall be requested to adjudicate the disagreements. The EM shall recommend an action mentioned under item “E” stated under “Duties & Responsibilities of EM (under section 5.4).

iv. In a dispute, following a thorough discussion, the decision of a Board of Examiners chaired by the Dean of the Faculty shall be the final subject to ratification by the Faculty Board and the Senate.

B. Double-Informed Marking (also called Double-Seen marking) 

The First Examiner marks, and then the Second Examiner marks the full cohort with the sight of the first marker’s marks and feedback. The mark is either confirmed or amended after discussion between markers. Double-Blind Marking is advised unless all scripts are marked by the second Examiner (which is beyond moderation).

Double-Informed Marking is recommended in

 i. reviewing the work of new or inexperienced First Examiner/Setter.

ii. dissertation /research projects

iii. as a solution for a severe discrepancy when a sample of scripts are marked by the Second Examiner (recommended by the External Moderator)

C. Sampled Moderation by Internal Moderators/Board of Moderation (Preliminary Results Board Chaired by the Chief Examiner)

The Internal Moderators/Moderation Board shall review a sample of marked scripts by the First Examiner/Panel of Marking Examiners & the        Second Examiner to confirm that the final marks/grades are consistent with the marking criteria and requirements. The sample must be meaningful and random to ensure requirements stated in Annexure 8.20.

The sample must include scripts/assessment work from

i. several distant grades, including failed candidates and those with A & A+

ii.     any individual candidate(s) where the examiners find significantly difficult to mark

At least 10%-15% of scripts and/ a minimum of 5 (maximum of 30) shall be considered in sampled moderation. The Chief Examiner shall                  determine the number.

The Internal Moderators/Moderation Board shall not generate a new mark for individual students. Instead, the Moderators/Board shall examine the     concerns listed in Annexure 8.20, having already generated marks, graphical distributions & descriptive statistics (mean, median, mode and SDs),       question papers, marking schemes/model answers, ILOs and Assessment Blueprint at hand. The moderation process should confirm that the               generated grades are valid and fair by all stakeholders. If concerns about marking standards or a systematic error in marking or marks processingare identified, those shall be communicated to the Chief Examiner through Annexure 8.20. These may be handled together if moderation done by a Moderation Board (pre-results) and actions would be taken instantly. Possible actions would vary depending on the concerns raised and may include further moderation, mark adjustments, complete remark of the entire cohort of scripts etc. Any adjustments to marks, if proposed,must be conducted systematically so that all affected work is treated equally and not just the moderated sample.

The Chief Examiner may perform the task of an internal moderator if the person is not a First /Second Examiner. External Moderator (=External        Examiner) shall be consulted or be a mandatory member of the Board when significant decisions are made. The moderation report and the      Examiners’ response (Annexure 8.20) should be documented and communicated to the Chair (Dean) at the Final Results Board, so the Board shall     decide whether to accept the response/treatment to marks/grades or require further actions at the Final Results Board.

Final Results Board

Once the Chief Examiner confirms the conformity of results to the requirement, the results shall be placed at the Final Results Board. Annexure         8.20 shall be tabled at the Final results Board.

In final year examinations, borderline cases may be considered in determining classes before finalising results during final moderation (Final             results Board) per the section 2.3.1.9 of this guide. Each faculty shall specify the guidelines for this purpose.

*, This step of moderation of marking shall be compulsory for all courses/modules that will contribute to the final GPA and the Class of the              degree awarded.

D. Benchmarking during team/panel marking

In panel marking, the Chief Examiner shall call a meeting prior to marking to review model answers, marking standards and processes in the              context of the relevant marking criteria. In this, a sample of assessments shall be marked by the Chief Examiner/one Examiner and share those            with the panel members to establish a benchmark (calibration). Or each marker may mark a sample of papers and then share the marking and               feedback with the Chief Examiner and other markers and achieve the calibration.

In moderation, to ensure concerns under Annexure 8.20 by the moderator/Board, the sample shall include scripts of candidates

i. with several distant grades, including failed candidates and those who received A & A+.

ii. any individual candidate(s) where the examiners find significantly difficult to mark.

iii. sample of scripts marked by each panel marker.

E.   Performances, presentations, or practical examinations

When assessments (with a summative value) involve performances, presentations or a demonstration of a practical/skill, especially in continuous assessments where results are released to students within the semester, such assessment tasks should be observed by two evaluators with prior agreed and informed marking rubrics. Later the marks should be moderated using the Annexure 8.20 at least by the Chief Examiner of the course/module with the assistance of the External Moderator if required before releasing results. When the employment of two evaluators is not possible, adjunct best practices such as moderation on student notes, slides, evaluator’s notes and audio-visual recordings are recommended.

F. Dissertations and research projects reports

Dissertation or report assessment shall be subject to full, independent, and anonymous first & second marking (or having two examiners going through the research/project report independently). The Supervisor may also contribute to marking on selected aspects. Each faculty shall have detailed guidelines on pertinent procedures involved.

G. Exemptions from moderation of marking

The following assessment work of students can be exempted from moderation.

i. Assessments that have only a formative value

ii. Assessments of non-GPA course (The Chief Examiner may decide)

iii. When marking is automated only the concerns of Annexure 8.20 shall be ensured in moderation

iv. In any other instance that the Senate of the University determines.

Notes on Moderation:

➢ Clerical checking and moderation are different

➢ Each faculty shall have a workflow on moderation (both for setting & marking) covering the specific requirements in line with this Manual and              maintain a Faculty Board approved policies and SOP on that.