Assessing Learning & Performance

Test, Measurement, Evaluation & Assessment

Test

Instrument/procedure (set of questions/problems, or task) that uses numerical scale or classification scheme (pass/fail or yes/no) for observing and describing individual’s characteristics
Usually test students …
1. ability (aptitude in subject)
2. effort (amount of work and time spent)
3. understanding (knowledge & comprehension)
4. cognition/thinking (application, analysis and evaluation)
5. psychomotor skills (manipulation & dexterity)
6. motivation (enthusiasm and inclination)

Measurement

Process of obtaining and assigning numerical value (score) or unitary quantity to specific attribute/characteristic of object, event or individual (e.g. a student) so as to allow comparisons with other objects, events or individuals

Evaluation

Process of making value judgement about quality of learner’s performance or product

Assessment

Process of obtaining information to make decisions about:
1. Students/learners
2. Curricula/programs
3. Educational policies

Approaches to Assessment

Assessment Objectives

Placement/Selection

To determine learner’s current ability, knowledge, skills, experience
To determine learner’s suitability for a course of study, instructional program, learning environment, job assignment, career path
To determine the appropriate mode of instruction for a learner

Curriculum decisions (formative)

Ways to improve a program/curriculum

Curriculum decisions (summative)

Effectiveness of the program/curriculum

Diagnostic

To identify students with learning difficulties
To determine causes of learners’ persistent or recurring learning difficulties

Types of Assessment

Formative assessment

To monitor learner’s progress during the course or program
To provide feedback to both trainer and learner
To diagnose learning difficulties
To plan and modify/adjust instructional program or teaching according to learners’ needs

Summative assessment

To provide final feedback to stakeholders about their achievement
To provide feedback to trainers or administrators about effectiveness of program
To assign grades/credits to students

Assessing Cognitive Domain

Old

New

Bloom's taxonomy

Bloom's taxonomy of learning objectives is used to assess students' ability to think at the different levels of cognitive complexity

Constructing Conventional Assessment Plan

Factors to consider

Explanations

Examples

Learning objectives

WHAT the learners are expected to achieve

To describe the structures of biomolecules and explain their functions

Timeframe

HOW LONG it will take to teach and assess topic

2 weeks

Formative assessment

ASSESS students' learning during teaching

Tutorial questions
Group discussion
Quizzes

Summative assessment

TEST students' learning (formal test) at the end of unit

1 hour paper containing short answer questions and essays

Constructing Table of Specifications (TOS) for Summative Assessment

Includes the following areas of assessment

Content topics
Thinking skills
Learning objectives
Weightings
- % of total test covered by topic
- Number of test items for topic

Planning for Assessment

Steps involved

WHEN: to be carried out
WHO: to discuss with
WHAT: to consider
- Scheduling
- Assessment modes
- Test items and contents
- Duration and frequency of testing
- Mark schemes
- Moderation

Effective Assessment Procedure

Reliability

Indication of the extent to which test/assessment measurements are consistent

Validity

Indication of the extent to which test/assessment truly measures what it is intended to measure

Reliable measurement may not be valid but valid measurement has to be reliable.

E.g. A weighing scale may be calibrated wrongly and give not valid measurements. If wrong readings are consistent, the measurements are still reliable.

Reliability

Context to measure consistency of results

Students carry out same task on repeated occasions
Students carry out two or more different but equivalent tasks on same or different occasions
More than one teacher marks the same tasks performed by the same students

Pearson correlation coefficient, r

Range of values: -1.00 to +1.00

Perfect positive correlation: r = 1.00 (e.g. age increase, wisdom increase)
No correlation: r = 0.00
Perfect negative correlation: r = -1.00 (e.g. age increase, eyesight decrease)

Reliability over time

Test-retest reliability (stability coefficient)

a. Calculate ‘r’ using scores on identical content samples tested on different occasions

i. r = 0 (scores are completely unreliable/inconsistent)

ii. r = 1 (scores are perfectly reliable)

b. Usually r lies around 0.8 to 0.9 for a standardized test within 6 months’ testing interval

Reliability over time

Delayed equivalent-forms reliability

a. Calculating ‘r’ using scores on equivalent content samples tested on different occasions

i. Correlation between two sets of scores is affected by both content and time interval

ii. Value of ‘r’ reflects both equivalence of assessment tasks and consistency of students’ performance

Reliability over content

Equivalent-forms reliability

a. Calculating ‘r’ using scores on equivalent or parallel content samples tested on same occasion, using same group of students

i. Correlation between two sets of scores is affected by difference in content

b. Value of ‘r’ reflects equivalence of assessment tasks

Reliability over markers

Inter-rater reliability

a. To assess scorers/markers in terms of:

i. Objectivity in marking

ii. Consistency

iii. Interchangeability

b. To obtain inter-rater reliability index,

i. Get two markers to mark the same sample of scripts

ii. Calculate correlation coefficient for the two sets of scores

iii. Strong correlation indicate high inter-rater consistency

Validity

Validity is achieved by matching test/assessment procedures to mastery and/or developmental learning objectives.

Mastery learning

o Assessment tasks or procedures should allow students to demonstrate the skill or knowledge specified in the learning objective or outcome

Developmental learning objectives

o Assessment tasks or procedures should allow students to demonstrate progress in attaining the learning objective

o Need to assess the same LO in different ways and at different stages of learning

o For example, learning how to use the microscope takes place step by step, teacher will check if students can complete step 1 before moving to step 2.

Criteria/Evidence of validity in assessment

Type

Definition

Features

How to obtain

Content

Extent to which the sample of content knowledge covered in test items represents the overall content knowledge

o How much of content knowledge covered in test items (overall content knowledge)

o Representative of what was being taught

o Material worthy (relevance and quality) of learning

o Assessment tasks should match learning objectives and curriculum contents

o Rating of assessment tasks (importance, accuracy, relevance and meaningfulness)

o Establish extent of representation and relevance of assessment tasks as a whole

Substantive

Extent to which assessment tasks involve the use of cognitive/thinking skills and processes

o Extent to which assessment tasks involve use of thinking skills and processes stated in curriculum

o Extent to which thinking skills and processes used by students are representative of cognitive skills intended to be assessed

o Determine cognitive skills required for successful performance (wrt TOS)

o Match taught thinking skills/strategies and those required in assessment tasks

o Analyse students’ cognitive strategies by getting them to ‘think aloud’

o Establish the relevance, desirability and representativeness of cognitive skills being assessed (let students know that for the particular thinking level, they will need to answer in a particular manner)

Practicality

Extent to which the implementation of the assessment procedure is feasible and cost effective

o Feasibility and cost effectiveness of using assessment procedure on intended number of students

o Ease of implementation – appropriate schedule, duration, competence of administers

o Teacher’s support for assessment procedure

o Extent of explaining individual differences (how much can it discriminate students)

o Extent of pinpointing students’ misunderstandings

o Need for replacement by alternative assessment procedures

o Conduct analyses of …

§ Cost of implementation

§ Efficiency and usefulness of instructions

§ Teacher feedback

§ Results from trial or pilot studies

Reliability

Extent to which results under different conditions (e.g. time, scorers) are consistent

o How similar assessment results would be if assessment procedure was conducted …

§ On different occasions over time

§ By different administrators, graders, and scorers

§ As a second, alternate form but with similar content

o Margins of error if differences arose

o Conduct studies to determine reliability of assessment results

o Statistical analyses

Analysis of Results

Raw score (no of points received on a test when the test is marked according to marking scheme) is not meaningful without further information either in criterion-referenced interpretation or norm-referenced interpretation.

Criterion-referenced interpretation

· It is a description on the specific tasks that a pupil can perform (pass-fail, bank marks)

· Uses percentage-correct score

· Most meaningful when a test is specifically designed to measure a set of clearly stated learning outcomes

· Absolute grading

Norm-referenced interpretation

· Derived/standardized scores indicate students’ relative position in a clearly defined referenced group

· A derived score is a numerical report of test performance on a score scale that has well-defined characteristics and yields normative meaning

· Relative grading

Motivational Styles

Adaptive motivational styles

Maladaptive motivational styles

Promotes ‘learning goals’ whereby individuals seek to increase their level of competence and understanding in a task

Fails to promote the establishment of realistic learning goals

Favours ‘performance goals’ characterised by the pursuit of positive judgements or avoidance of negative judgements of competence

Mastery oriented

Self-worth motivated

Learned helplessness

· Perceive their failure experiences as a ‘problem’ that can be remedied and is within their control

· View their success as due to their own ability

· View effort as a tool to overcome failure and achieve success

· Maintain their level of self-confidence in spite of failure experiences

· Tend to establish and maintain a positive self-image

· Need for personal and social acceptance

· When confronted with a difficult task, they need to minimise risk-taking in order to ‘look good’ or ‘save face’ and to prevent others from believing that they lack ability

· Attribute their failure to factors beyond their control

· Attribute their achievements to situational factors such as luck and ease of task

· Develop a lowering of expectancy of success, an inclination towards failure acceptance

· View success as a less rewarding experience

· Consider the need for effort in a task as a proof of their lack of ability

Test Anxiety

Increased in emotional tension brought about by the prospect of an impending assessment/evaluation

Sarrason's factors of test anxiety

Cognitive

Emotional

Task-irrelevant thinking

Worry

Tension

Physical reactions

· Self-preoccupation

· Failure-centeredness

· Helplessness

· Escapism

· Troubled over nature of task

· Fret about doing badly

· Inability to focus on task

· Bothered about other’s performance

· Feel jittery

· Distressed, uneasy

· Short-tempered

· Moody

· Headache

· Sweaty palms

· Palpitations

· Loss of appetite, queasy

Types of test-anxious students

1. ill-prepared for the test

2. don’t understand their work

3. good in their work but fear failure

4. lack good study skills

Reducing test-anxiety

1. training students to be test-wise (train their test-taking skills)

2. giving specific instructions to concentrate on test items rather focusing on anxiety

3. ensure that auditory and visual distractors are absent from test environment

4. habituation / increasing exposure to test conditions

5. ensure test content is covered before test

6. arrange test items in order of difficulty-level (starting with least difficult and ending with most)

7. ensure that test items are of appropriate level of difficulty

8. give detailed feedback to learners regarding their performance, hence show them how to improve

Google Sites

Report abuse