Appendix E- SAM ‘Self-Assessment of MTSS Implementation” Rubric

North Carolina Self-Assessment of MTSS Implementation (SAM) Item Descriptors

The Self-Assessment of MTSS Implementation (SAM), now in its second edition, measures school-level implementation of MTSS. The purpose of administration is to help school-level and district-level personnel identify and prioritize implementation steps. The SAM contains 39 items in 6 domains (Leadership, Building Capacity/Infrastructure for Implementation, Communication and Collaboration, Data-based Problem-solving, Three-tiered Instructional/Intervention Model, and Data-Evaluation). The SAM was originally developed in Florida, and has undergone a national pilot for use in other locations.

For use in North Carolina, a standard setting project for the SAM was also conducted. A diverse group of educational professionals experienced and skilled in the implementation of multi- tiered, data-based support systems (e.g., Responsiveness to Instruction, Positive Behavior Intervention and Support, MTSS) were utilized to set this criterion. In addition, this expert panel also reviewed each item on the SAM to determine its accuracy and validity for use in North Carolina. In order to add additional robustness to the assessment of MTSS implementation, the expert panel also identified existing school-level and district-level work productsthat would be used as evidence in the administration of the SAM.

When will it be used? As a self-report and guide for school teams in implementation, the SAM can be used at any time. However, one time per year (April-June is the recommended time frame), the district MTSS coordinator and/or another member of the MTSS District Team would facilitate administration at the school. This facilitated administration would allow the district personnel to review evidence to support the school team s proposed score.

Directions for annual administration as a fidelity measure:

1. Each team member should review the SAM item descriptors and think how s/he, personally, would respond to each item.

2. After reviewing the SAM item descriptors independently, the team members should come together with the district MTSS coordinator and/or member of the MTSS district team to discuss their responses and reachagreement on which answer best represents the current status of implementation at their school.

3. The district personnel facilitating the administration will use the suggested evidence below each item at their discretion to verify the school team s responses on the SAM.

4. The school team, with the help of the district personnel can use the data to plan best next steps for MTSS implementation.

5. Total scores for the facilitated SAM administration will produce one of three levels of implementation within each domain: not implementing, initially implementing, or fully implementing.

In order to receive ascorewithlevelof implementationfor eachdomain, thefacilitator is responsiblefor enteringscorelevelswithinthe NCSAMExcel Scoring Protocol

Adapted from The Self-Assessmentof MTSS. (2013). Florida’s MTSS, Florida Department of Education. Adapted with permission

57

NC'SAM'Version'1,'October,'2015'

'

SAM Item and examples of supporting evidence Not Implementing Emerging/Developing Operationalizing Optimizing

1: The principal is actively involved in and facilitates

MTSS implementation

! School Improvement Plan shows evidence of MTSS systems and practices

! Agendas and meeting rosters showing evidences of data- based problem-solving

! PD Plan(s) with MTSS systems and practices showing principal involvement

! Staff/student handbook with evidence of MTSS practices

2: A leadership team is established that includes 5I7 members cross-disciplinary representation (e.g., principal, general and special education teachers, content area experts, student support personnel1) and is responsible for facilitating MTSS implementation2

! Leadership team roster

! Leadership team meeting agendas/minutes

3: The leadership team actively engages staff in ongoing professional development and coaching3 necessary to support MTSS implementation

! Professional development and coaching plan

! Professional Development roster(s)

4: A strategic plan for MTSS implementation is developed and aligned with the school improvement plan4

! MTSS implementation/strategic plan with alignment to or as a part of the School Improvement Plan

5: The leadership team is actively facilitating implementation of MTSS6 as part of their school improvement planning process

! School improvement plan with evidence (direct language or components explicitly mentioned) of MTSS

The principal does not actively support MTSS.

No leadership team with explicit responsibility for leading MTSS implementation exists

The leadership team does not have a needs-based plan to provide staff with professional development or coaching to support MTSS implementation

No strategic plan for MTSS implementation exists

The leadership team is not actively engaging in efforts to facilitate MTSS implementation

The principal communicates an urgent desire to implement MTSS, participates in professional development on MTSS, and is establishing an MTSS vision

A leadership team exists that includes cross- disciplinary representation,

A needs assessment is conducted to gather information on beliefs, knowledge, and skills to develop a professional development plan to support MTSS implementation

Leadership team is engaging district, family, and community partners to identify stakeholder needs, resources for and barriers to MTSS implementation

The leadership team engages in action planning and has created a strategic plan to facilitate implementation of the

critical elements7 of MTSS

AND the principal actively supports the leadership team and staff to build capacity for implementation

AND the leadership team has explicit expectations for facilitating MTSS implementation,

AND a professional development plan is created based on the needs assessment and used to engage staff in ongoing professional development and coaching

AND as part of the school improvement planning process a strategic plan is developed that specifies

MTSS implementation5

AND the leadership team provides support to educators implementing the critical elements of MTSS identified in the strategic plan

AND the principal actively supports data-based problem-solving use at the school

AND the leadership team members have the beliefs, knowledge, and skills to lead implementation efforts

AND ongoing professional development activities are informed by data collected on the outcomes of professional development and coaching for continuous improvement

AND a strategic plan for MTSS implementation is updated as needed based on student outcome and implementation fidelity data as part of the school

improvement planning process

AND the leadership team uses data on

implementation fidelity of the critical elements of MTSS to engage in data- based problem-solving for the purpose of continuous school improvement

Adapted from The Self-Assessmentof MTSS. (2013). Florida’s MTSS, Florida Department of Education. Adapted with permission.

Adapted from The Self-Assessmentof MTSS. (2013). Florida’s MTSS, Florida Department of Education. Adapted with permission

Adapted from The Self-Assessmentof MTSS. (2013). Florida’s MTSS, Florida Department of Education. Adapted with permission. 63

NC SAM Version 1, October, 2015

Adapted from The Self-Assessmentof MTSS. (2013). Florida’s MTSS, Florida Department of Education. Adapted with permission

64

NC SAM Version 1, October, 2015

Adapted from The Self-Assessmentof MTSS. (2013). Florida’s MTSS, Florida Department of Education. Adapted with permission

65

NC SAM Version 1, October, 2015

Adapted from The Self-Assessmentof MTSS. (2013). Florida’s MTSS, Florida Department of Education. Adapted with permission

66

NC SAM Version 1, October, 2015

Adapted from The Self-Assessmentof MTSS. (2013). Florida’s MTSS, Florida Department of Education. Adapted with permission

67

NC SAM Version 1, October, 2015

SAM Item and examples of supporting Not Implementing Emerging/Developing Operationalizing Optimizing

DataIEvaluation

34: Staff understand and have access to academic, behavior and social-emotional data sources that address the following purposes of assessment: 1) identify students at-risk academically, socially, and/or emotionally, 2) determine why student is at-risk, 3) monitor student academic and social-emotional growth/progress, 4) Inform academic and social- emotional instructional planning, 5) determine student attainment of academic/behavioral outcomes

! Assessment Plan (within or separate from MTSS implementation plan)

! Assessment inventory

35: Policies and procedures for decision- making are established for the administration of assessments, access to existing data sources, and use of data

! Assessment inventory

! School Improvement plan

! Progress-monitoring data

36: Effective data tools are used

Staff do not understand and have access to academic, behavior, and social-emotional data sources that address the purposes of assessment

No policies and procedures are in place

Staff do not have access to

Staff learn the purposes of assessment within MTSS and the leadership team selects measures for the purposes of assessment across academic, behavior and social- emotional areas that are reliable, valid and accessible, as well as culturally, linguistically, and developmentally appropriate

The leadership team outlines policies and procedures for decision- making that include schedules for screening, use of diagnostic assessments, progress monitoring frequency, and criteria for determining tier(s) of support needed

The leadership team ensures availability of tools that can track

AND staff engage in assessment with fidelity to:

1) answer predetermined guiding/critical questions

regarding student functioning/outcomes, 2) identify students who are at-risk at least 3I4 times/year, 2) determine why a student is at risk,

3) monitor student growth/progress, 4) inform instructional/intervention

planning, 5) determine student attainment of academic, behavior, and social-emotional outcomes

AND staff consistently administer assessments, access data sources and make data-based decisions using policies and procedures for decision- making with fidelity

AND staff use the data tools and are provided

AND the leadership team and/or staff collaboratively and systematically evaluate and adjust assessment practices to ensure availability of accurate and useful data to inform instruction, and assessment tools are evaluated for continued value, usefulness, and cultural, linguistic, and developmental appropriateness

AND adherence to and effectiveness of policies and procedures for decision making are evaluated regularly for efficiency, usefulness, and relevance for students and staff, and data are used to make adjustments to the policies

AND data tools are periodically assessed and

appropriately and independently by staff

tools that efficiently provide and graphically display academic,

assistance as needed

the necessary changes are

! Assessment Plan (within or separate from implementation plan)

! Graphing results

! Professional Development/Coaching plans on data tools use

data needed to answer problem solving questions for academic, behavior and social- emotional issues

behavior and social-emotional data, and staff are trained on the use of the tools and on their responsibilities for data collection, entry and management

made in order to improve

functionality, efficiency, and usefulness, and staff is proficient and independent with data tools and easily support new staff members

Adapted from The Self-Assessmentof MTSS. (2013). Florida’s MTSS, Florida Department of Education. Adapted with permission.

68

NC SAM Version 1, October, 2015

1 Instructional support staff may include: interventionists, coaches, behavioral specialists, etc. Student support personnel are comprised of school psychologists, school counselors, social workers, school nurses, etc.

2 Responsibilities for facilitating MTSS implementation are not limited to, but can include:

o Promoting a schoolwide vision and mission for MTSS implementation, including the development and dissemination of a schoolwide implementation plan

o Allocating resources (e.g., time, personnel, materials) for the planning and delivery of evidence-based assessment, instruction and intervention

o Providing ongoing professional development and coaching support to school staff

o Collecting and analyzing data on MTSS implementation efforts

3 Professional development and coaching are ongoing activities that develop the capacity of staff to implement MTSS. Efforts should be aligned with results of school needs assessments and modified based on the results of professional learning.

4 At the school level, a school-based leadership team should guide implementation of a MTSS. This may take place within the structure of the School Improvement Team or may be a subset of this team that is charged with implementation planning. Teams may differ based on several factors but a connection should always be made in order to facilitate

Adapted from The Self-Assessmentof MTSS. (2013). Florida’s MTSS, Florida Department of Education. Adapted with permission. 69

NC SAM Version 1, October, 2015 '

effective implementation. A long-term plan for implementation of MTSS should be developed by the school based leadership team. This may be a part of the school improvement plan or separate from it but again should be aligned with the overall goals and actions within the school improvement plan.

5 A strategic plan for MTSS implementation should address the following components (at a minimum): a. Communication and collaboration strategies; b. Capacity building targets and activities;

c. Data to monitor implementation fidelity of the critical components of MTSS

6 Different approaches to facilitating school-wide implementation of an MTSS model can include:

The focus on a three-stage model of consensus building, infrastructure development, and implementation of practices consistent with an MTSS model The focus on a specific sets of activities related to successful implementation of a designated model of service delivery (e.g., National Implementation Research Network framework)

The approach to facilitating school-wide implementation of an MTSS model should be connected to the School Improvement Plan (SIP), as well as other school-wide plans.

7 Critical elements of MTSS communicated to staff include: Curriculum standards; Assessment data used to inform instruction; Multiple tiers of instruction and intervention; Data-based problem solving used to make decisions

8 “Coaching” is defined as technical assistance and support provide to school staff to improve implementation of components of an MTSS model, including: Co- Planning, Modeling/Demonstration, Co-Facilitation, and Guided practice with high quality feedback. “Coaching does NOT necessarily have to be completed by one person. Coaching can be provided by a number of different individuals depending upon their specializations, skill sets, as well as the particulars of the context of activities. It is unreasonable to assume that just one individual, or one coach will have all the skills required to effectively provide coaching for MTSS in every given situation that may arise.” March, A.L. and Gaunt, B.T. (2013). Systems Coaching: A model for building capacity.

9 Behavior/Social-Emotional Assessment: Screening: Recommended Behavior/Social-Emotional screening data include reviewing and analyzing all students adherence to school-wide expectations through collection of:

o Minor problem behavior (classroom managed)

o Major problem behavior (office discipline referral)

o Attendance patterns

o Other areas that some schools may choose to universally screen in the area of Behavior/Social-Emotional skills using a school-wide screening for internalizing behaviors (e.g., depressive symptoms, anxiety, etc.).

Diagnostic: Diagnostic assessments for behavior/social-emotional skills include use of functional behavior assessments in order to find the root cause for the student s difficulties. Progress-Monitoring: In the area of behavior/social-emotional functioning, the monitoring of student progress with the intervention should be matched with the problem of concern. Within progress-monitoring of behavior, teams will want to consider monitoring frequency, duration, intensity and latency recording.

10 Schools will need to establish and communicate the problem solving process to be used, specific steps to be followed, and criteria to use when making decisions (e.g., what is good, questionable, or poor response to instruction/intervention). Schools should consider district and state guidelines when available.

70

'

11 Processes and procedures for problem solving, data collection and use, and decision-rules include:

o Specific guidelines on the steps of problem solving to be used

o Documentation requirements

o Opportunities for engaging in data-based problem solving (e.g., Professional Learning Communities, etc)

o Roles and responsibilities of participants

12 Resources encompass not only available monetary assets but also available personnel, instructional materials, and time that will facilitate the implementation and sustainment of an MTSS as a framework for supporting all students.

13 Staff refers to employees at the school that will be impacted by or will be involved in implementation of MTSS. This will always include administration, teachers, other professionals and para-professional support staff. The degree to which other employees (e.g., bus drivers, cafeteria workers, administrative support staff, etc.) are included may be determined by their level of involvement with/implementation of MTSS components at the individual school level.

14 Efforts to engage staff should align with district and state guidance regarding MTSS implementation to facilitate staff understanding of connections between school, district and state initiatives.

15 Data on student outcomes, school-level implementation fidelity, the capacity of educators to implement, and commitment from staff are needed to inform implementation. Staff roles and responsibilities will drive the specific data they need to inform implementation.

16 Family and community engagement is the active and meaningful partnership that educators build and maintain with students families and the broader community for the purpose of supporting student learning.

17 Intensive outreach to unresponsive families refers to additional activities undertaken by the school to engage families of students who need additional supports, but who are not engaging with the school s typical outreach practices (e.g., letters and phone calls home). Intensive outreach is an individualized approach requiring information gathering and problem solving to identify outreach strategies that are more likely to be successful for a family.

18 Data-based problem solving refers to a multi-step process that includes examining performance related to goals/expectations (problem identification), understanding variables causing problems (problem analysis), selecting/designing and implementing strategies to lessen barriers and achieve goals (instruction/intervention delivery), and monitoring effectiveness (monitoring/evaluation).

19Data-based problem solving should occur (a) across content areas (reading, math, science, behavior, social-emotional and other relevant content areas for a school) (b) within and across grade levels (e.g., horizontal meetings for 6th, 7th, 8th, as well as vertical meetings), and (d) across tiers (performance data in response to instruction used to engage in problem solving for all students (Core), for some students receiving supplemental instruction (Supplemental), and for students receiving individualized support (Intensive).

20 Reasons why students are not meeting expectations are sometimes referred to as hypotheses or barriers to learning. The big idea is that schools identify potential curriculum, instruction, environmental (e.g., peer distractions, classroom management issues), and learner (e.g., skill deficits) for why the student is not meeting expectations and collect data/information to determine which reasons are contributing to the problem.

71

21 Specific instruction/intervention plans include information outlining:

o The goal of the intervention/action plan

o What intervention or action steps (e.g., curriculum adjustments, instructional processes and procedures) will be put in place

o How often (daily/weekly/etc.) the intervention will be utilized

o How long each session is to be completed

o Who is responsible for intervention implementation and support

o Where and when the intervention will happen

o Plan for monitoring instruction/intervention fidelity and progress towards identified goals

o Timeframe (dates) for periodic review of progress monitoring data and decision points

22 Structured problem solving is utilized to identify resources that can be used to facilitate implementation and barriers that are hindering implementation for the purpose of developing specific action plans to increase implementation levels.

23 Priority learning standards are curriculum standards that define what students should know and be able to do for a given content area and grade level (e.g., NCSCOS, Social- Emotional/Behavior Standards, etc.).

24 Expectations for instruction often include elements related to the instructional routine (e.g., whole-group, small-group, and independent practice), amount of time dedicated to instruction, and which evidence-based instructional strategies are used.

25

Both statewide assessments and formative assessments administered to all students are important to identify so that expectations for the data needed to inform

decisions are consistent.

26 Structured instruction of behavioral expectations and social and emotional skills is provided to all students. Classroom routines include social and emotional learning principles and classroom management strategies embedded into instruction. School climate and environments support student well-being. A small number of clearly defined school-wide expectations that are positively stated are a foundational element of a Tier One school-wide behavior support system.

27 School-wide social-emotional behavior data may include Office Discipline Referrals, In-School Suspensions, Out-of-School Suspensions, and social-emotional screening data sources used to examine theeffectiveness of Tier One behavior and social-emotional supports.

28 Tier two interventions should be aligned with Tier One instructional goals and expectations, address high-probability barriers to achieving instructional goals and expectations, and include assessments, which measurespecific skills, general outcomes, and student progress.

29 Tier Two interventions should be aligned with school-wide behavior and social-emotional expectations, address high-probability barriers to meeting instructional goals and student well-being, and include assessmentsthat monitor student discipline incidents, social-emotional skills, and well-being.

Adapted from The Self-Assessmentof MTSS. (2013). Florida’s MTSS, Florida Department of Education. Adapted with permission. 72

NC SAM Version 1, October, 2015

30 Tier three interventions generally provide increased exposure (time in minutes) to quality instruction or intervention, more focused instruction matched to student need, and smaller groupings. Additionally, Tier Three interventions often are developed during individual student focused problem solving sessions. Importantly, Tier Three interventions focused on academic issues should be linked to Tier One and Two instructional content and processes and also should consider what behavioral and social-emotional supports are needed for success.

31 Tier Three interventions are matched to a student s specific behavior and social-emotional needs and ensure the student has access to Tier One and Tier Two supports. For a few students with complex needs,individualized interventions may involve wraparound supports across systems (e.g., mental health, education, medical, family, etc.).

Individualized interventions include specific prevention and consequence-based strategies based on assessment information (i.e., Functional Behavior Assessment), and may include modifications to the classroom environment or instruction, teaching new skills, and reinforcement of desired behaviors as well as a range of supports such as mental health services.

Adapted from The Self-Assessmentof MTSS. (2013). Florida’s MTSS, Florida Department of Education. Adapted with permission. NC SAM Version 1, October, 2015