Glossary of Terms

AA-AAAS = Alternate Assessment based on Alternate Academic Achievement Standards for students with the most significant cognitive disabilities.

Academic Achievement Indicator = methodology for calculating its Academic Achievement indicator consistent with the requirements under ESEA section 1111(c)(4)(B) and (E).

Must an SEA adjust its Academic Achievement indicator based on participation rate for assessments administered for the 2021-2022 school year?

Yes. Each SEA is required to annually measure the achievement of not less than 95 percent of all students and students in each subgroup on its annual statewide assessments in reading/language arts and mathematics under ESEA section 1111(b)(2)(B)(v)(I).
For purposes of measuring, calculating, and reporting on the Academic Achievement indicator, the denominator must be the greater of 95 percent of all students, or 95 percent of all students in the subgroup; or the number of students participating in the assessments under (ESEA section 1111(c)(4)(E)).
Academic Achievement indicator uses individual student performance on annual state tests for both general and alternate Math and English Language Arts content standards.

Academic Growth Indicator = An academic growth indicator uses individual student performance on annual state tests for the general and alternate content standards over between two consecutive years to determine the amount of improvement of a student’s proficiency on grade level Math and ELA standards over the two-year period.

Accessibility – Degree to which the items or tasks on a test enable as many test takers as possible to demonstrate their standing on the target construct without being impeded by characteristics of the item that are irrelevant to the construct being measured. The purpose of accessibility supports is to eliminate the effects of student barriers that prevent them from demonstrating what they know and can do. All students can work toward grade level academic content standards via differentiation and appropriately selected supports and accommodations.

Accessibility features – universal tools, designated supports, and/or accommodation values.

Accommodation – Adjustments that do not alter the assessed construct that are applied to test presentation, environment, content, format (including response format), or administration conditions for particular test takers that are embedded within assessments or applied after the assessment is designed. Accommodated scores should be sufficiently comparable to unaccommodated scores that they can be aggregated together. These supports increase equitable access during the assessment and generate valid assessment results for students for whom there is documentation of need on an Individualized Education Program (IEP) or 504 (Plan), and in the ACT may include English Learner (EL) Plan.

Accountability System = The accountability system uses multiple measures of school success, including academic outcomes, student progress, and school quality, while emphasizing that all students deserve a high-quality and well-rounded education that will prepare them for success. Montana’s Consolidated State Plan under Every Student Succeeds Act.

Accurate – The data are “correct” and free of errors. Fundamentally important for all data.

Accreditation – Certification by the Montana Board of Public Education (Board) that a school meets the adopted standards of the Board for a specified school year.

Adaptive Behavior – Actions essential for someone to live independently and to function safely in daily life.

Adaptation/test adaptation – 1. Any change in test content, format (including response format), or administration conditions that is made to increase the test accessibility for individuals who otherwise would face construct-irrelevant barriers on the original test. An adaptation may or may not change the meaning of the construct being measured or alter score interpretations. An adaptation that changes score meaning is referred to as a modification; an adaptation that does not change the score meaning is referred to as an accommodation.

Adjusted Participation Rate – All students, including students with disabilities and English Learners, are required to take part in the state assessments with or without accommodations. Students are expected to take part in state assessments in one of three ways: (1) Participate in the general education assessments without accommodations. (2) Participate in the general education assessments with accommodations. (3)Participate in alternate assessments when the participation criteria are met. The participation rate must be calculated separately for each subject (i.e., reading/language arts, mathematics, and science). Students who are medically exempt from the assessment are not included in the adjusted participation rate. For 2020-2021 and 2021-2022 this includes all COVID Reason Exemptions.

Advanced level (Level 4) – means superior performance (ARM 10.54.2502).

Alternative Assessment – An alternative assessment is an assessment based on alternative achievement standards which is designed for students with the most significant cognitive disabilities who cannot participate in the general assessment even with appropriate accommodations. This is a way to measure the performance of students who are unable to participate in general large-scale assessments even with accommodations. The student’s Individualized Educational Program (IEP) team makes the determination of whether a student is able to take the regular assessment. Note: These assessments are intended for students with the most significant cognitive disabilities. For more information about the assessment, read the MontCAS Policies and Procedures for Participation in State Assessments, Alternate Assessment Eligibility Guidelines, and the Participation Form.

ALTPARTALTACH = This is code used in federal reporting via EdFacts to denote the student participated in the alternate assessment based on alternate achievement standards.

Annual Meaningful Differentiation Process = An SEA has the discretion to adjust the weighting of indicators, as long as the academic indicators (i.e., the Academic Achievement, Other Academic, Graduation Rate, and Progress in Achieving ELP indicators), in the aggregate, have much greater weight than the School Quality or Student Success (SQSS) indicators, in the aggregate.

Assessment – Gathering, organizing, and evaluating information about student learning in order to monitor and measure student attainment of a specific set of content standards, and the effectiveness of the instructional program (ARM 10.55.602).

Asterisk = On the OPI's state data dashboards, an asterisk (*) indicates this number has been suppressed for student privacy and security reasons (Privacy and Security Details) or the data was either not available or could not be reported. For more information about why data may not be available or reporting, visit the Report Card Information Page.

Audit – Activities the state education agency conducts to ensure local assessments administered by the local educational agency as defined.

Authorized Representative (or AR) – individual designated by a State or local educational authority headed by an official listed in 34 CFR §99.31(a)(3) to conduct – with respect to Federal – or State-supported education programs – any audit or evaluation, or any compliance or enforcement activity in connection with Federal legal requirements that relate to these programs. For more information, see the Family Educational Rights and Privacy Act regulations, 34 CFR §99.3. Further the OPI has the following policy:

- The authorized representative is initially established by OPI policy depending on the relative size of the school district. The role may be reassigned to the Board Chair or

other representative of the organization by official school board action, as described later in this section.

- The authorized representative for a school district receiving federal and/or state grant funds is:
  - - the district superintendent; or
    - if there is no district superintendent, the principal; or
    - if the district has neither a district superintendent nor a principal, the county superintendent; or
    - the Board Chair or other individual, based on an exception approved by the OPI.

Benchmark – means expectations for a student's knowledge, skills, and abilities along a developmental continuum in each content area. That continuum is focused at three points: the end of grade 4, the end of grade 8, and upon graduation (grade 12) (ARM 10.54.2502).

Breach – (1) an event, intentional or not, that results in the inappropriate exposure of test items or answers that could potentially impact the accuracy of the test results; OR (2) an action by others before, during, or after a test administration to impact student test scores (e.g., educators changing student answer sheets).

Building Coordinator (or BC) – This role is assigned by the AR and/or STC and is typically a licensed non-instructional person. Typically, staff holding this role are principals, vice principals, technology coordinators, counselors, or other staff members. It is highly recommended this role is reserved for people with non-instructional or limited instructional duties so that they can coordinate and monitor testing activity in the school. The STC can furnish the Building Coordinator with access to the secure online portals and for any staff below this role, the Building Coordinator can create user accounts. Principals are also able to report non-participation for medical reasons in the MontCAS Application if assigned this activity by the STC.

Chain of Custody – the chronological documentation or paper trail that shows the custody, control, and transfer of assessment materials.

Cheating – The most common type of test fraud and it is is any action which jeopardizes the overall reliability and validity of score interpretations.

Comparability (score comparability) – the degree of score comparability resulting from the application of a linking procedure varies along a continuum that depends on the type of linking conducted. See alternate forms, equating, calibration, linking, moderation, projection, and vertical scaling.

Compromise – disclosure of test items or forms; can be intentional or unintentional; may also refer to changing the interpretation of a test score or changing the test score itself.

Confidence – Disclosure of test items or forms; can be intentional or unintentional. May also refer to changing the interpretation of a test score or changing the test score itself.

Confidentiality – State of keeping or being kept secret or private.

Consistent – In order for data to be meaningful over time, consistency in methodologies and practices are paramount, especially in reporting where data may be compared.

Construct – Concept or characteristic the test is designed to measure.

Content domain – The set of behaviors, knowledge, skills, abilities, attitudes or other characteristics to be measured by a test, represented in detailed test specifications, and often organized into categories by which items are classified.

Content standard – In educational assessment, a statement of content and skills that students are expected to learn in a subject matter area often at a particular grade or at the completion of a particular level of schooling. It is what all students should know, understand and be able to do in a specific content area at the end of grade [ARM 10.53.103]. This term is synonymous with academic achievement standards in this text.

Computer Adaptive Testing (CAT) – A form of computer-based test that adapts to the student’s ability level.

Computer-Based Testing (CBT) – a test taken by a student on a computer and scored by a computer.

Conflict of Interest – applied to any person who handles assessment materials or student data who could be perceived as having a special interest in a particular student or group of students, such as a parent.

Credibility – The quality of being trusted and believed in.

Data Forensics – The use of analytic methods to identify or detect possible cheating. Procedures can include evaluation of score gains, aberrance or person fit, erasures, latency analysis, similarity analysis, and examination of changes in student responses (wrong-to-right, right-to-wrong, wrong-to-wrong).

Detection – Monitoring, reporting, and working with the OPI when irregularities are found.

Deviation – A citation of noncompliance with any given standard.

Disclosure – Disclosure means to permit access to or the release, transfer, or other communication of personally identifiable information (PII) by any means (34 CFR §99.3). Disclosure can be Authorized, such as when a parent or an eligible student gives written consent to share education records with an authorized party (e.g., a researcher). Disclosure can also be Unauthorized or inadvertent (accidental). An unauthorized disclosure can happen due to a data breach or a loss, and an accidental disclosure can occur when data released in public aggregate reports are unintentionally presented in a manner that allows individual students to be identified. In the event secure test items have been disclosed means that secure assessment items or materials have been shared with unauthorized persons or parties.

Directory Information – Directory information is information contained in the education records of a student that would not generally be considered harmful or an invasion of privacy if disclosed. Typically, "directory information" includes information such as name, address, telephone listing, date and place of birth, participation in officially recognized activities and sports, and dates of attendance. A school may disclose "directory information" to third parties without consent if it has given public notice of the types of information which it has designated as "directory information," the parent's or eligible student's right to restrict the disclosure of such information, and the period of time within which a parent or eligible student has to notify the school in writing that he or she does not want any or all of those types of information designated as "directory information." (see 34 CFR § 99.3 and 34 CFR § 99.37).

Double-Testing – The act of requiring a student to participate in more than one form of the required end of year state summative assessment which is designed to measure the specific grade-level content standards in a given year.

Early Stopping Rule (or ESR) - If a student’s responses to test items are not clearly observable or understood by the Test Administrator or scribe, the testing experience may need to be ended early using the Early Stopping Rule.

Eligibility – A student who satisfies the appropriate criterion or conditions to participate in certain assessments.

Eligible Student Population = All students enrolled during the testing window whether or not they were present for the full academic year except students who did not participate in the assessments due to significant medical emergencies (see 34 CFR 200.2(a)(b)(1)(i)).

English Learner = A student who has been impacted linguistically by their environment, who is identified as having limitations in their social and academic English language proficiency, and whose English language proficiency denies them the ability to meet the challenging state academic standards or the ability to fully participate and successfully achieve in classrooms where the language of instruction is English. Defined under the OPI’s guidance as English Learner Guidance for School Districts, English Learner Checklist for Districts, and the Standardized Entrance-Service-Monitor-Exit Procedures for ELs.

English Learner Progress Indicator = English Learner Progress refers to evaluating English Learners' (EL) growth from one point in time to another. It considers how language growth impacts their mastery of the regular curriculum and graduation rates. Based on research, the OPI has determined that students growing at least 0.5 points on the composite score on the World Class Instructional Design and Assessment (WIDA) access test each year should attain English language proficiency within five years. The OPI’s definition of progress is any student who shows a 0.5-point growth on their composite score from the previous year.

Exposed – Secure assessment items or materials have been viewed.

Fair –

Falsifying – alter (information or evidence) so as to mislead.

FERPA (or Family Educational Rights and Privacy Act) – Federal law that protects the privacy of student education records. The law applies to all schools that receive funds under an applicable program of the U.S. Department of Education. FERPA gives parents certain rights with respect to their children's education records. These rights transfer to the student when he or she reaches the age of 18 or attends a school beyond the high school level. Students to whom the rights have transferred are "eligible students." For more information, visit https://studentprivacy.ed.gov/.

Fidelity – The degree of exactness with which something has been given.

Field test – test items that are in the final stages of development and are being monitored for quality by being administered to a sample group.

Fraud – Whether intentional or unintentional in nature, any behavior that threatens the validity of the test score is considered as test fraud.

Generalizable –

Identification – The action or process of identifying a student for specific assessments and services.

IEP – Individual Education Programs

Inference – A conclusion reached on the basis of evidence and reasoning – typically from information gathered in a test about student’s knowledge, skills or abilities.

Integrity – The quality of upholding strong standards. As presented in the NCME white paper from Greg Cizek (2012), test integrity emphasizes that valid testing requires the results to be useful, interpretable, accurate, and comparable. The technical merits of the test scores must meet industry standards with respect to fairness, reliability, and validity; however, cheating and security breaches can pollute the data, reducing or eliminating their value.

Inspection (other places it is investigation) - Focused on a multiple-measure approach of collecting evidence and working toward a resolution.

Invalidation – An act of omitting test results and student responses from the testing, reporting, and accountability systems for a given testing event for which the student may not retest.

Irregularity – includes many different activities - not necessarily cheating, but anything unusual that happened during testing (e.g., the fire alarms going off or a power outage).

Item – A statement, question, exercise, or task on a test for which the test taker is to select or construct a response, or perform a task. See prompt.

Meaningful – Having a serious, important, or useful quality or purpose.

Medical Exemption – A policy adopted by the OPI and defined within the MontCAS Policies and Procedures for Participation in State Assessments to address students who are unable to participate in the state assessment during the testing and make-up windows because of a significant medical emergency. These students cannot be assessed at any time during the testing window due to a significant medical emergency. Generally, if a student can receive instruction, then they are also able to participate in state assessments. In extraordinary circumstances (e.g., medical emergency), individual students may be exempt in the participation rate calculations for the purpose of determinations under the accountability process. Non-participation for any other reason will negatively impact the adjusted participation rate calculation.

MEDEXEMPT = This is code used in federal reporting via EdFacts to denote the student did not participate due to an OPI approved medical exemption. Medically exempt students cannot be counted as “non-participants”; accordingly, such students are in the counts of students not expected to test.

Misconduct – Misbehavior during testing, such as inappropriate proctoring or other violations of the standard testing protocol.

Modification is any practice or procedure that compromises the intent of the assessment through a change in the learning expectations, construct, or content that is to be measured, grade-level standard, or measured outcome of the assessment that is not authorized explicitly by the OPI. Any change away from a standard administration that is not listed in the OPI’s assessment-specific accessibility and accommodation guidelines can be considered a modification.

Monitor – Activities the OPI and school districts use to ensure compliance with test administration and test security requirements.

Nearing Proficiency Level (Level 2) – means a student has partial mastery of the prerequisite knowledge and skills fundamental for proficient work at each benchmark (ARM 10.54.2502).

Non-routine accommodation – Accommodations not routinely approved are not published as available in the Usability, Accessibility, and Accommodations Guidelines for the specific state assessment. However, students may require accommodations for students that are identified as having a disability where a specific accommodation is specified in the student’s IEP/Section 504 Plan. In this event, the OPI must approve all non-routine accommodations that are used in the state assessments before a student can receive them as them may violate the construct of interest leading to invalid scores. This policy ensures students, parents, teachers, and schools are aware that some accommodation decisions may impact accountability and reporting. For reporting this low incidence needs or for more information, System Test Coordinators can access the Non-Routine Request for Accommodation Page inside the MontCAS Application.

Non-Participant [NPART] = This is code used in federal reporting via EdFacts to denote the student did not participate. This is a student who was expected to test but did not for whatever reason and their non-participation status will count negatively against the school’s participation rate (e.g., parent refusal or student absence).

Novice Level (Level 1) – means a student is beginning to attain the prerequisite knowledge and skills that are fundamental for work at each benchmark (ARM 10.54.2502).

Observable Response – Predictable and consistent behavior or movement that is able to be understood by a communication partner as intentional communication.

Participant = must have (1) a valid score and be (2) assigned a proficiency level. (see 34 CFR 200.2(a)(b)(1)(i)). All students, including students with disabilities are required by state and federal law to take part in the OPI’s state assessments with or without accommodations. All students are expected to take part in state assessments in one of three ways: (1) Participate in the general education assessments without accommodations. (2) Participate in the general education assessments with accommodations. (3) Participate in Alternate Assessments when the participation criteria are met.

Participation – The action of taking part in state end-of-year summative assessment.

Passage – see Prompt. This is a specific kind of “stimulus” and has been used interchangeably with the term “stimulus”. The distinction derives from Smarter Balanced requirements in the ELA test to acknowledge potential construct validity needs and the appropriate application of student supports access the stimulus on the ELA test.

Performance Levels – means the level of achievement in broad, general terms such as Advanced Level (Level 4); Proficient Level (Level 3); Nearing Proficiency Level (Level 2); and Novice Level (Level 1) (ARM 10.54.2502).

Performance standard – means the specific expectations for performance in each content area at each of the three benchmarks. Performance standards define the quality of performance and describe the performance to be demonstrated (ARM 10.54.2502).

Prevention – Following best practices for the test integrity and security aspects of the design, development, operation, and administration of state assessments (both paper/pencil and online) to prevent irregularities from occurring.

Proctor – A person who monitors students during an examination.

Proficient Level (Level 3) – means solid academic performance for each benchmark, reaching levels of demonstrated competency over challenging subject matter, including subject-matter knowledge, application of such knowledge to real-world situations, and analytical skills appropriate to the subject matter (ARM 10.54.2502).

Prohibited Materials/Devices – any electronic devices used for communication, capturing images of the test or testing room, or data storage (e.g., smartphones, smart watches, cell phones, book readers, electronic tablets, pagers, cameras, non-approved calculators, music players, voice recorders, etc.) that can disrupt the testing environment, and be used to compromise the security and confidentiality of the test. Students should not have any prohibited devices near them during testing. Failure to comply with this policy may result in score invalidations for individual students or groups of students depending on the degree the security, reliability, or validity of the assessment has been compromised.

Prompt: – The text or source about which the item is written that elicit a test taker’s response. The student reads/views/listens to it and then responds to the item/task question(s). Typically represented with a graphic, piece of text, table, or chart that is associated with multiple test items.

Quality – The standard of something as measured against other things of a similar kind; the degree of excellence of something.

Regular Assessment - An assessment designed to measure the student’s knowledge and skills in a particular subject matter based on academic achievement standards appropriate to the student’s grade level (see ESEA, Section 1111(b)(2)). Note: Regular assessments based on grade level academic achievement standards can be taken with or without accommodations.

REGPARTWOACC = This is code used in federal reporting via EdFacts to denote the student Participated in regular assessment based on grade-level achievement standards without accommodations

REGPARTWACC = This is code used in federal reporting via EdFacts to denote the student participated in regular assessment based on grade-level achievement standards with accommodations

Reliability - Refers to the extent to which assessments are consistent and measure with great confidence the same things for valid interpretations of student performance from one child to another and over time.

Replicable – Able to be copied or reproduced exactly.

Resolution – Working together to investigate irregularities and resolve issues to ensure valid results for all students.

Retake –

Roster –

Register –

Secure – Materials cannot be handled, viewed, or any contents disclosed.

Security – Protecting or safeguarding.

Scribe – A scribe is an adult who writes down what a student dictates in a variety of ways (e.g., speech, American Sign Language (ASL), braille, assistive communication device). (See Smarter Balanced Scribing Protocol.)

School Official – any employee, including teacher, that the school or district has determined to have a “legitimate educational interest” in the personally identifiable information from an education record of a student. School officials may also include third party contractors, consultants, volunteers, service providers, or other party with whom the school or district has outsourced institutional services or functions for which the school or district would otherwise use employees under the school official exception in FERPA. Additional information about the Family Educational Privacy Act is available at 34 CFR § 99.31(a)(1).

Special Population Students = Students with disabilities (IEP and/or 504) and current English Learners.

Stimulus – see Prompt.

Standard – Normal expectation of technical quality and accepted practice or process.

Standardized Testing Environment - Designated in-school area for state test administration that provides an environment that minimizes distractions and disruptions for students.

Students with Significant Cognitive Disability[ies] – Meets the four-participation criterion as described Appendix A. The student has records that indicate intellectual disability[ies] that significantly impact intellectual functioning and adaptive behavior. Note: Not all students with these disabilities are considered to have a “significant cognitive disability.” Students with significant cognitive disabilities require substantial modifications, adaptations, or supports to meaningfully access the grade-level content aligned to the standards. A significant cognitive disability will be pervasive, affecting student learning across content areas and in social and community settings. The student has demonstrated cognitive abilities and adaptive behavior and cannot participate in the general education curriculum even when provided with accommodations and requires a modified curriculum linked to the standards. The student requires direct and extensive individualized instruction in order to acquire, maintain, generalize, and transfer new skills. Having a significant cognitive disability is not determined by an IQ test score, but rather a holistic understanding of the student. Verifiable evidence will demonstrate the impact of the significant cognitive disability in all aspects of the student’s life [see Appendix A]. Determinations for student participation in state assessments must be data-centered and made individually for each student by the IEP team.

Student Records – Includes (a) the name and address of the student; (b) his/her parent or guardian; (c) birth date; (d) academic work completed; (e) level of achievement (grades, standardized achievement tests); (f) immunization records as per 20-5-406, MCA; (g) attendance data; and (h) the statewide student identifier assigned by the Office of Public Instruction. The local board of trustees shall establish policies and procedures for the use and transfer of student records that are in compliance with 20-1-213, MCA, and state and federal laws governing individual privacy. For more information, see ARM 10.55.909.

System Test Coordinator (or STC) – Since the early 2000s, the STC is a role that has been used at the OPI. This person is assigned by the AR and by OPI’s definition is the sole person responsible for managing the administration of the suite of MontCAS assessments that meet federal accountability requirements under the Elementary and Secondary Education ACT and state law (ARM 10.56). This person is the OPI's single point of contact for all assessment-related communications, the data steward for managing local users within the restricted web-based assessment reporting systems used to administer each test, and the local liaison for all standardized testing procedures (i.e., proctoring, safeguarding, securing, and reporting state test information).

Test Administration – The range of activities involved, from receiving access to secure assessment materials, to handling materials, to the return of secure assessment materials to the OPI or its agents.

Test Administrator (or TA) – is assigned by the STC and/or BC and is a licensed educator. The OPI advises that classroom teachers are assigned as TAs. The OPI further recommends that the Test Administrator is familiar to the students, so students feel more comfortable in the testing environment. Teachers and Principals are essential for motivating these students to do their best. Schools that implement these strategies and encourage students tend to have higher student participation and engagement. Note: All Test Administrators must be trained and certified to administer the test and cannot have any personal conflicts with serving in this role for the students assigned to them.

Test fraud – A set of activities that are illegal, inappropriate, or against the rules/standards.

Testing Alert –

Testing Incident –

Timely – All data has specific timeframes and deadlines for collection and reporting. This is especially important where the data collected is a requirement for funding or integral to a “snapshot”, or point in time picture of the data.

Trusted –

Validity – Refers to the accuracy of the assessment - whether or not it measures what it is supposed to measure for appropriate inferences about students’ characteristics, usually based on observations of the student’s performance such as a test score.

Valued – Important or beneficial.

Variance to testing – An approved alternate approach to meeting or exceeding the minimum standardized procedures to test administration.

Veracious – Data collections should be conducted with fidelity and via sound practices that are free from manipulation or unauthorized modification or entry. Fidelity is particularly critical to financial and student assessment and performance data.

Web Monitoring – A process that can be used to address the risk to tests and items posed by illicit discussion, distribution, and sale of test content on the Internet. Web monitoring/patrol leverages technology tools and human expertise to identify, prioritize, and monitor websites, discussion forums, peer-to-peer servers, etc., where sensitive test information may be disclosed or at risk of disclosure.

Note: Some of these terms are defined in Administrative Rules for Montana (ARM). See ARM 10.55.602 DEFINITIONS. For a list of other terms not defined here, please visit the NCME Assessment Glossary at https://www.ncme.org/resources/glossary.