PTS

Objectives

At the end of the session, you should be able to:

• Recognize different assessment instruments

• Construct evidence plan, written test, table of specification, performance test and questioning tool.

• Develop a good assessment instrument

Introduction

Developing assessment instruments follows after we prepare the learning materials. These assessment instruments are important for both trainer and trainees. It is the tool used to verify trainees’ competency and this is what we use to gauge effectiveness of our session plan and learning materials.

In this lesson, we will focus not only on familiarizing template but more importantly on preparing them.

Preparing Evidence Plan

In developing evidence plan, we need to identify the evidence to gather. Evidence is the information gathered that serves as proof of competency, and has two forms – direct evidence (observed or witnessed by the assessor) and indirect evidence (reviewed or examined by the assessor). It must be valid, sufficient, current and authentic.

These evidences must be the scope in preparing the evidence plan. Evidence Plan is a document checklist that identifies evidence requirements and assessment methods to be used. It is a guide for the assessor to make sure that the appropriate method of assessment will be executed and will produce all the needed evidence in assessing the candidate.

Procedure in preparing Evidence Plan:

1. Review Assessment Criteria in Module of Instruction.

2. List down all evidence requirements taking note of the following:

   • All critical aspects of competencies included

   • Evidence Requirement for Four (4) Dimensions of Competency

   • Required Knowledge and Skills

3. Obtain the template.

4. Write the evidence requirements on the first column.

5. Identify evidence gathering methods to use and place it in designated columns.

6. Place a check mark to specify what assessment method to use for each evidence requirements. To establish corroboration of evidence, use at least two methods per criteria.

7. Review the output or consult other experts.

Preparing Evidence Gathering Tool

Evidence Gathering Tool or Assessment Tool refers to both the instrument and the instructions for gathering and interpreting evidence.

Examples of evidence gathering tool are written test, questioning tool and performance test that will be further discussed in this lesson.

Preparing Table of Specification

“Hinuhulaan lang naman ng titser natin ang exam natin” (Our teacher is just guessing the content of the exam), or “Hindi naman natin napag-aralan yan, bakit kasama sa exam?” (We haven’t studied that topic, why is it included in the exam?) are some of the rants of students before and after taking the exam. One strategy a teacher can use to rule out student’s doubt, increase consistency (reliability) of exam and make accurate (valid) measurement is by preparing a table of specification.

Table of Specification (TOS), also called test blueprint, is an instrument used to align objectives, instruction and assessment1. This serve as basis in test construction.

It has six (6) major elements intended to develop TOS for a comprehensive examination, which includes2:

• Balance among goals selected for examination

• Balance among levels of learning

• The test format

• The total number of items

• The number of items for each goal and level of learning

• The enabling skills to be selected from each goal framework

Procedure in preparing Table of Specification for a UNIT TEST

1. Obtain and review evidence plan to determine what evidence gathering tool to use.

2. Review Assessment Criteria in Module of Instruction.

3. Obtain the template.

5. Compute for the percentage of class time per Learning Outcome by dividing Nominal Hours with Nominal Duration, then multiply it to 100.

For example:
To get the percentage of class time of LO1, divide 4.5 Nominal Hours to 40 hrs. Nominal Duration (or total number of Nominal Hours).
4.5 / 40 = 0.1125
Then multiply it to 100 to get its percentage.
0.1125 * 100 = 11.25% 

8. Compute for the number test items by multiplying the desired/total number of test item by percentage of class time.

For example:
To get the number of test items for LO1, multiply 50 (or total number of test items) to 11.25% (percentage of class time).
50 * 11.25% = 5.625
To compute manually, multiply 50 by 11.25 then divide it by 100.
50 * 11.25 = 562.5 / 100 = 5.625  

Preparing Written Test

Written Test refers to the print or non-print evidence gathering tool used to measure trainees’ knowledge, skill and attitude. True or false, multiple choice, matching exercise, completion items, and essay exam are examples of types of test.

Characteristics to consider in developing good test:

1. Validity refers to how well a test measures what it is purported to measure1. The test should match the assessment criteria written in module of instruction.

2. Reliability refers to the degree to which an assessment tool produces stable and consistent results1.

3. Fairness refers to the state of being true or factual, and not based on trainer’s preference.

Procedure in preparing written test:

1. Planning for the test:

1. • Outline the content and identify the learning outcome

   • Review Competency Standard & Evidence Plan

   • Prepare the table of specification

   • Choose appropriate type of test (e.g. multiple choice, true or false, matching items etc.)

2. Preparing the test:

   • Choose the items to be included according to the table of specification

   • Construct the items based on the rules for the type/s chosen. Refer to the suggested reading, pp. 9 – 26

   • Arrange the grouping and sequence of items

   • Write the direction for the test and decide on the method of scoring

3. Analyzing and revising the test:

   • Perform test analysis to determine difficulty, discrimination and reliability Retain or discard items based on the analysis outcomes. Refer to suggested reading, pp. 4 – 8

Preparing Performance Test

Aside from the written test, a trainer can also use Performance Test to effectively measure the most critical aspect of each learning outcomes. In CBT, performance test or demonstration is a standard test to determine competence.

Performance Test is an evidence gathering tool used to assess trainee’s skills based on authentic tasks. The actual performance of competency is the primary source of evidence a trainer needs to gather to certify that the trainee has acquired the target technical skills. This type of test is used in demonstration and on-the-job observation.

To collect such evidence, listed below are the guidelines in preparing Performance Test: 

• Keep the general instruction detailed yet concise. It should contain the standard expected and the allotted time for the performance to be done

• Work performance, written as Specific Instruction, must be evident and measurable

• List the complete and exact available equipment, supplies, materials and tools

• Note that Performance Criteria Checklist will be used after each performance test as an assessment tool

Performance test enables the trainer to assess the procedure (or sequence of steps), the trainee’s level of competency and the outcome based on standards.

Preparing Interview Questions

Questioning Tool is essential in expounding competencies not evidently demonstrated in written and performance test. It is used in probing the breathe and depth trainees’ answers to ensure validity and reliability of assessment result.

Guidelines in Preparing Interview Questions:

• Questions can be derived from the following:

  1. Dimensions of competency such as task, task management, contingency management and job or role and environment management skills

  2. Vocational or technical knowledge

  3. Workplace procedure and regulations

  4. Safety requirements

• Keep the question brief, structured, and focused on one key concept

• Use language at the suitable level of the candidate

• Use open-ended probing questions to clarify, confirm or supplement evidence by asking questions “what would you do if...”, “what if you were using... instead of..?” or “how do you...?” rather than closed-ended questions

• Write a list of responses to ensure reliability of assessment