Reliability
Reliability
Are my assessment tasks reliable? Do they have a clear marking schemes and rubric?
1. Outstanding Reliability
o Assessment tasks and conditions are strategically designed to remove all sources of non-relevant variation in measurements.
o It is a high priority that assessment conditions are clear, consistent, and enforced.
o In units with multiple classes and teachers, consistency around messaging and assistance is ensured through embedded practice.
o Marking schemes and rubrics are clear and unambiguous to ensure consistency in student and marker interpretation.
o Consistency of marking is ensured through a range of moderation processes such as single marker of task or sub-task, double marking or sample double marking, utilising sample scripts/responses for all grade levels, or comprehensive in-school moderation or marking calibration activities.
o Instructions/questions are clear and unambiguous to student interpretation.
2. High Reliability
o Assessment tasks and conditions are thoughtfully designed to remove sources of non-relevant variation in measurements.
o Assessment conditions are clear and do not advantage or disadvantage individual students.
o In units with multiple classes and teachers, consistency around messaging and assistance is considered.
o Marking schemes and rubrics are clear and aim to reduce marker variation.
o Consistency of marking is considered through processes such as single marker of task or sub-task, double marking or sample double marking, access to sample scripts/responses, or in-school moderation or marking calibration activities.
3. Satisfactory Reliability
o Assessment tasks and conditions are designed with some consideration of reducing sources of non-relevant variation in measurements.
o The assessment and assessment conditions are discussed in units with multiple classes and teachers.
o There is a marking scheme developed for the task and applied in marking.
o Different markers discuss marking of concern to ensure consistency and have access to an answer key or sample answers.
4. Minimal Reliability
o Assessment tasks are designed with minimal consideration of reducing sources of non-relevant variation in measurements.
o Assessment conditions could be interpreted differently by different students.
o There is minimal discussion about the assessment between teachers of the same unit.
o The marking scheme is underdeveloped and requires interpretation.
o Different markers barely discuss the marking, or the answer key or sample answers are underdeveloped or incomplete.
5. No Reliability
o Performance in the assessment tasks is largely determined by sources of non-relevant variation.
o Assessment conditions are not clearly stipulated to students and could be interpreted very differently by different students.
o There is minimal discussion about the assessment between teachers of the same unit. There is no clear marking scheme.
o There is no answer key or similar provided. Different markers do not discuss marking.