Assessment Instruments

C- Assessment Instruments

A- Criterion-referenced instruments
B- Student work samples
C- Student Narratives
D- Performance Standards and Assessment Rubrics

A- Criterion or objective-referenced assessment instruments:
(Learner-Centered) p145
> Assessment that enhances student learning (Baron, 1998)

Criterion-referenced assessments are

• linked to instructional goals

• linked to objectives derived from goals

The purpose of this type of assessment is to evaluate:

1. students' progress: they enable learners to reflect on their own performances > they will become ultimately responsible for the quality of their work

2. instructional quality: they indicate to the designer which components of the instruction went well and which need revision

+ it contains a criterion that specifies how well a student must perform the skill in order to master the objective.
>Therefore, performance criteria should be congruent with the objectives, learners, and context.

The performance required in the objective must match the performance required in the test. (p 146)

Two uses of criterion-referenced tests:

• pretests: they have 2 goals

  • to verify that the student possesses the anticipated entry behaviors

  • to measure the student's knowledge of what is to be taught.

• posttests: they are used primarily to measure the student's knowledge of what was taught.

Types of Criterion-Referenced Tests & Their Uses (p.146)

a- Entry behaviors test: It is given to learners before they begin instruction in order to assess those learners' mastery of prerequisite skills.
This entry behavior test should cover the skills that are more questionable than others in terms of being already mastered by the target population.
From entry behaviors test scores designers decide whether learners are ready to begin the instruction.

b- Pretest: It is given to learners before they begin instruction in order to determine whether they have previously mastered some or all of the skills that are to be included in the instruction.
If all the skills have been mastered, then the instruction is not needed.
The pretest includes one or more items for key skills identified in the instructional analysis, including the instructional goal.
Since both entry behaviors tests and pretests are administered prior to instruction, they are often combined into one instrument.
From pretest scores designers decide whether the instruction would be too elementary for the learners and, if not too elementary, how to develop instruction most efficiently for a particular group.
A pretest is valuable only when it is likely that some of the learners will have partial knowledge of the content.

c- Practice/rehearsal tests: They are focused at the lesson rather on the unit level. The purpose for practice tests is to:
- provide active learner participation
- enable learners to rehearse new knowledge and skills
- enable learners to judge for themselves their level of understanding and skill
- enable the professors to provide corrective feedback
- enable the professors to monitor the pace of instruction

d- Posttests: They are administered following instruction, and they are parallel to pretests (excluding entry behaviors items).
Posttests measure objectives included in the instruction. They assess all of the objectives, and especially focus on the terminal objective

Designing a Test (p.149)

A criterion-referenced test is designed by matching the learning domain with an item or assessment task type.

a- Objectives in the Verbal Information Domain: they require objective-style test items which include the following formats:

• short answer

• alternative response

• matching

• multiple-choice items

b- Objectives in the Intellectual Skills Domain: they require one of the following:

• objective-style test items

• the creation of a product (essay, research paper...)

• a live performance of some type (act in a play, make a presentation, conduct a business meeting...)

If an objective requires the learner to create a unique solution or product, it would be necessary to 

1. write directions for the learner to follow

2. establish a set of criteria for judging response quality

3. convert the criteria into a checklist or rating scale (rubric) that can be used to assess those products

c- Objectives in the Affective/Attitudinal Domain: They are concerned with the learner's attitudes or preferences. Items for attitudinal objectives require one or both of the following:

• that the learners state their preferences

• that the instructor observes the learners' behavior and infers their attitudes from their actions

d- Objectives in the Psychomotor Domain: They are usually sets of directions on how to demonstrate the tasks and require the learner to perform a sequence of steps that represents the instructional goal. Criteria for acceptable performances need to be identified and converted into a checklist or rating scale that the instructor uses to indicate whether each step is executed properly.

Writing Test Items (p. 151)

There are 4 categories of criteria that should be considered when creating test items:

1. Goal-Centered Criteria: The test items and assessment tasks should be congruent with the terminal and performance objectives; they should provide learners with the opportunity to meet the criteria necessary to demonstrate mastery of an objective.
   
   No rule states that performance criteria should or should not be provided to learners. Sometimes it is necessary for they to know performance criteria and sometimes it is not.
   
   (It is very beneficial for the learner to receive the criteria before s/he is given the test; this will help them to prepare for it much more efficiently)

1. Learner-Centered Criteria: The test items and assessment tasks must be tailored to the characteristics and needs of the learners. Criteria in this area include considerations such as learners'

   1. vocabulary and language levels

   2. developmental levels (for setting task complexity)

   3. motivational and interest levels

   4. experiences and backgrounds

   5. special needs
      
      Designers should consider how to aid learners in becoming evaluators of their own work and performances. Self-evaluation and self-refinement are two of the main goals of all instruction since they can lead to independent learning.

2. Context-Centered Criteria: The test items and assessment tasks must be as realistic or authentic to the actual performance setting as possible. This criterion helps to ensure transfer of the knowledge and skills from the learning to the performance environment.

3. Assessment-Centered Criteria: The test items and assessment tasks must include clearly written and parsimonious directions, resource materials, and questions + correct grammar, spelling, and punctuation!
   
   To help ensure task clarity and to minimize learners' test anxiety, learners should be given all the necessary information to answer a question before they are asked to respond.
   
   Items should not be written to trick learners! Ideally, learners should err because they do not possess the skill, not because the test item or assessment is convoluted and confusing!

Evaluating Tests and Test Items (p.156)

When writing test directions and test items, the designer should ensure the following:

1. test directions are clear, simple, and easy to follow

2. each test item is clear and conveys the intended information

3. conditions under which responses are made are realistic

4. the response methods are clear to learners

5. appropriate space, time, and equipment are available

Using Portfolio Assessments (p. 162)

Portfolios are collections of criterion-referenced assessments that illustrate learners' work. These assessments might include:

• objective-style tests that demonstrate progress from the pretest to the posttest

• products that learners developed during instruction

• live performances

• assessments of learners' attitudes about the domain studied or the instruction

There are several features of quality portfolio assessment:

1. the work samples must be anchored to specific instructional goals and performance objectives

2. the work samples should be the criterion-referenced assessments collected during instruction (the pretests and posttests)

3. each assessment is accompanied by its rubric with a student's responses evaluated and scored, indicating the strengths and problems within a performance.

The assessment of growth is accomplished at two levels:

1. learner self-assessment: learners examine their own materials and record their judgments about the strengths and problems + what they might do to improve the materials

2. instructor assessment: instructors examine the materials set without examining the evaluations by the learner, and record their judgments.

Then both the learner and the instructor compare their evaluations , discussing any discrepancies between the two evaluations.

As a result, they plan together next steps the learner should undertake to improve the quality of his/her work.

B- Student work samples:

• written reports

• rough drafts

• notes

• revisions

• projects

• Journals

• peer reviews

• self-evaluations

• anecdotal records

• class projects

• reflective writings

• artwork; graphics

• photographs

• exams

• computer programs

• presentations

 C- Student Narratives: A narrative description that

• states goals

• describes efforts

• explains work samples

• reflects on experience.

D- Performance Standards and Assessment Rubrics:

A variety of scoring criteria can be used in developing performance standards or assessment rubrics. Standards and rubrics should include:

• performance levels (a range of various criterion levels)

• descriptors (standards for excellence for specified performance levels)

• scale (range of values used at each performance level)

Examples of Rubric:

• analytic assessment rubric

• holistic assessment rubric: it provides a general score for a compiliton of work samples rather than individual scores for specific work samples