Glossary of Terms

A

Accommodation – Changes in procedures or materials that increase equitable access during the assessment and generate valid assessment results for students for whom there is documentation of need on an Individualized Education Program (IEP) or 504 (Plan) or English Learner (EL) Plan.

Accreditation – Certification by the Montana Board of Public Education (Board) that a school meets the adopted standards of the Board for a specified school year.

Accurate – The data are “correct” and free of errors.  Fundamentally important for all data.

Assessment – Gathering, organizing, and evaluating information about student learning in order to monitor and measure student attainment of a specific set of content standards, and the effectiveness of the instructional program (ARM 10.55.602). 

Audit – Activities the state education agency conducts to ensure local assessments administered by the local educational agency as defined.

Authorized Representative (or AR) – individual designated by a State or local educational authority headed by an official listed in 34 CFR §99.31(a)(3) to conduct – with respect to Federal – or State-supported education programs – any audit or evaluation, or any compliance or enforcement activity in connection with Federal legal requirements that relate to these programs. For more information, see the Family Educational Rights and Privacy Act regulations, 34 CFR §99.3.  

B

Breach – (1) an event, intentional or not, that results in the inappropriate exposure of test items or answers that could potentially impact the accuracy of the test results; OR (2) an action by others before, during, or after a test administration to impact student test scores (e.g., educators changing student answer sheets).

Building Coordinator (or BC)  – This role is assigned by the AR and/or STC and is typically a licensed non-instructional person. Typically, staff holding this role are principals, vice principals, technology coordinators, counselors, or other staff members. It is highly recommended this role is reserved for people with non-instructional or limited instructional duties so that they can coordinate and monitor testing activity in the school. The STC can furnish the Building Coordinator with access to the secure online portals and for any staff below this role, the Building Coordinator can create user accounts. Principals are also able to report non-participation for medical reasons in the MontCAS Application if assigned this activity by the STC.  

C

Comparability (score comparability) – the degree of score comparability resulting from the application of a linking procedure varies along a continuum that depends on the type of linking conducted. See alternate forms, equating, calibration, linking, moderation, projection, and vertical scaling

Chain of Custody – the chronological documentation or paper trail that shows the custody, control, and transfer of assessment materials.

Cheating – The most common type of test fraud and it is is any action which jeopardizes the overall reliability and validity of score interpretations. 

Compromise – disclosure of test items or forms; can be intentional or unintentional; may also refer to changing the interpretation of a test score or changing the test score itself.

Confidence – Disclosure of test items or forms; can be intentional or unintentional. May also refer to changing the interpretation of a test score or changing the test score itself.

Confidentiality – State of keeping or being kept secret or private.

Consistent – In order for data to be meaningful over time, consistency in methodologies and practices are paramount, especially in reporting where data may be compared.

Content standard – What all students should know, understand and be able to do in a specific content area.

Computer Adaptive Testing (CAT) – a form of computer-based test that adapts to the student’s ability level.

Computer-Based Testing (CBT) – a test taken by a student on a computer and scored by a computer.

Conflict of Interest – applied to any person who handles assessment materials or student data who could be perceived as having a special interest in a particular student or group of students, such as a parent.

D

Data Forensics – The use of analytic methods to identify or detect possible cheating. Procedures can include evaluation of score gains, aberrance or person fit, erasures, latency analysis, similarity analysis, and examination of changes in student responses (wrong-to-right, right-to-wrong, wrong-to-wrong).

Detection – Monitoring, reporting, and working with the OPI when irregularities are found.

Deviation – A citation of noncompliance with any given standard.

Disclosed – Secure assessment items or materials have been shared.

Double-Testing – The act of requiring a student to participate in more than one form of the required end of year state summative assessment which is designed to measure the specific grade-level content standards in a given year. 

E

Eligibility – A student who satisfies the appropriate criterion or conditions to participate in certain assessments.

Early Stopping Rule (ESR) – The Early Stopping Rule applies to some alternate assessments, and is used to end a test if a student does not have an observable mode of response to the assessment. 

Exposed – Secure assessment items or materials have been viewed.

F

Falsifying  –  alter (information or evidence) so as to mislead.

FERPA (or Family Educational Rights and Privacy Act  – Federal law that protects the privacy of student education records. The law applies to all schools that receive funds under an applicable program of the U.S. Department of Education. FERPA gives parents certain rights with respect to their children's education records. These rights transfer to the student when he or she reaches the age of 18 or attends a school beyond the high school level. Students to whom the rights have transferred are "eligible students." For more information, visit https://studentprivacy.ed.gov/.

Fidelity – The degree of exactness with which something has been given.

Field test – test items that are in the final stages of development and are being monitored for quality by being administered to a sample group.

Fraud – Whether intentional or unintentional in nature, any behavior that threatens the validity of the test score is considered as test fraud.

G

H

I

Identification - The action or process of identifying a student for specific assessments and services.

Inference – A conclusion reached on the basis of evidence and reasoning – typically from information gathered in a test about student’s knowledge, skills or abilities.

Integrity – The quality of upholding strong standards. As presented in the NCME white paper from Greg Cizek (2012), test integrity emphasizes that valid testing requires the results to be useful, interpretable, accurate, and comparable. The technical merits of the test scores must meet industry standards with respect to fairness, reliability, and validity; however, cheating and security breaches can pollute the data, reducing or eliminating their value.

Inspection (other places it is investigation) - Focused on a multiple-measure approach of collecting evidence and working toward a resolution.

Invalidation – An act of omitting test results and student responses from the testing, reporting, and accountability systems for a given testing event for which the student may not retest.

Irregularity – includes many different activities - not necessarily cheating, but anything unusual that happened during testing (e.g., the fire alarms going off or a power outage)

J

K

L

M

Misconduct – Misbehavior during testing, such as inappropriate proctoring or other violations of the standard testing protocol.

Monitor – Activities the OPI uses to ensure compliance with test administration and test security requirements.

N

O

P

Participation – The action of taking part in MontCAS summative assessments.

Prevention – Following best practices for the test integrity and security aspects of the design, development, operation, and administration of state assessments (both paper/pencil and online) to prevent irregularities from occurring.

Proctor (PR) – a person who monitors students during an examination.

Q

Quality  – The standard of something as measured against other things of a similar kind; the degree of excellence of something.

R

Reliability - Refers to the extent to which assessments are consistent and measure with great confidence the same things for valid interpretations of student performance from one child to another and over time.

Replicable  – Able to be copied or reproduced exactly.

Resolution – Working together to investigate irregularities and resolve issues to ensure valid results for all students. 

S

Secure – Materials cannot be handled, viewed, or any contents disclosed.

Security – Protecting or safeguarding.

Scribe – A scribe is an adult who writes down what a student dictates in a variety of ways (e.g., speech, American Sign Language (ASL), braille, assistive communication device). (See Smarter Balanced Scribing Protocol.)

School Official  –  any employee, including teacher, that the school or district has determined to have a “legitimate educational interest” in the personally identifiable information from an education record of a student. School officials may also include third party contractors, consultants, volunteers, service providers, or other party with whom the school or district has outsourced institutional services or functions for which the school or district would otherwise use employees under the school official exception in FERPA. Additional information about the Family Educational Privacy Act is available at 34 CFR § 99.31(a)(1).

Standard – Normal expectation of technical quality and accepted practice or process.

Standardized Testing Environment –  designated in-school area for state test administration that provides an environment that minimizes distractions and disruptions for students.

Student Records –  Includes (a) the name and address of the student; (b) his/her parent or guardian; (c) birth date; (d) academic work completed; (e) level of achievement (grades, standardized achievement tests); (f) immunization records as per 20-5-406, MCA; (g) attendance data; and (h) the statewide student identifier assigned by the Office of Public Instruction. The local board of trustees shall establish policies and procedures for the use and transfer of student records that are in compliance with 20-1-213, MCA, and state and federal laws governing individual privacy. For more information, see ARM 10.55.909.

Student Response Check (SRC) –  The Student Response Check is used during some alternate assessments to determine a student's mode of observable response. 

System Test Coordinator (or STC) – This person is assigned by the AR and by OPI’s definition is the sole person responsible for managing the administration of the suite of MontCAS assessments that meet federal accountability requirements under the Elementary and Secondary Education ACT and state law (ARM 10.56). This person is the OPI's single point of contact for all assessment-related communications, the data steward for managing local users within the restricted web-based assessment reporting systems used to administer each test, and the local liaison for all standardized testing procedures (i.e., proctoring, safeguarding, securing, and reporting state test information).

T

Test Administration – The range of activities involved from receiving access to secure assessment materials to handling materials to the return of secure assessment materials to the OPI or its agents.

Test Administrator (or TA) – is assigned by the STC and/or BC and is a licensed educator. The OPI advises that classroom teachers are assigned as TAs. The OPI further recommends that the Test Administrator is familiar to the students, so students feel more comfortable in the testing environment. Note: All Test Administrators must be trained and certified to administer the test and cannot have any personal conflicts with serving in this role for the students assigned to them. 

Test fraud – A set of activities that are illegal, inappropriate, or against the rules/standards.

Testing Alert – A message delivered to the OPI from the assessment vendors when the system identifies words or phrases in a student's answer to a test item that may indicate there is a concern for the student. When an alert is received by the OPI, the school is contacted by the OPI to share the alert details so the school can follow up appropriately with the student and/or student's family.

Testing Incident – Whether intentional or by accident, failure to comply with security rules, either by staff or students, constitutes a test security incident. The OPI recognizes three levels of testing incidents: Level I, Level II, and Level III. Incidents Levels I–III are behaviors prohibited either because they give a student an unfair advantage or because they compromise the secure administration of the MontCAS assessments. 

Timely – All data has specific timeframes and deadlines for collection and reporting.  This is especially important where the data collected is a requirement for funding or integral to a “snapshot”, or point in time picture of the data.

U

V

Validity – Refers to the accuracy of the assessment - whether or not it measures what it is supposed to measure for appropriate inferences about students’ characteristics, usually based on observations of the student’s performance such as a test score.

Variance to testing – An approved alternate approach to meeting or exceeding the minimum standardized procedures to test administration.

Veracious – Data collections should be conducted with fidelity and via sound practices that are free from manipulation or unauthorized modification or entry.  Fidelity is particularly critical to financial and student assessment and performance data.

W

Web Monitoring – A process that can be used to address the risk to tests and items posed by illicit discussion, distribution, and sale of test content on the Internet. Web monitoring/patrol leverages technology tools and human expertise to identify, prioritize, and monitor websites, discussion forums, peer-to-peer servers, etc., where sensitive test information may be disclosed or at risk of disclosure.

X

Y

Z

Note: Some of these terms are defined in Administrative Rules for Montana (ARM). See ARM 10.55.602 DEFINITIONS. For a list of other terms not defined here, please visit the NCME Assessment Glossary at https://www.ncme.org/resources/glossary.