In this project, we have been using the ‘Work of Teaching Science’ (WOTS) framework  (Mikeska, Kurzum, Steinberg, & Xu, 2018) to guide the development of our CKT assessment instrument about matter and its interactions. The full WOTS framework:

• focuses on seven instructional tools (e.g., scientific models and representations; scientific investigations; etc.) that elementary science teachers interact with.

• includes 27 science teaching practices that research has indicated or hypothesized are critical for beginning elementary science teachers to know how to do well to be effective practitioners.

• is organized into two to six science teaching practices per instructional tool category (e.g., the scientific explanations tool category includes two science teaching practices — one about critiquing student generated explanations and another about selecting explanations of scientific phenomena).

To create each CKT assessment item, we used content-specific resources, such as research studies, educative curriculum guides and materials, and practitioner literature and knowledge, to generate instructional scenarios that involve elementary science teachers in leveraging their subject matter knowledge as they engage in these specific science teaching practices. This work involved identifying the content challenges elementary teachers face when teaching science within each of five key topic areas within matter and its interactions:

• properties of matter and their measurements;

• changes in matter;

• conservation of matter;

• model of matter; and

• materials.

These challenges include the work teachers do to identify and select instructional activities, science phenomena, demonstrations, and models used within each of these topic areas. The instructional scenarios were intended to directly map onto the science teaching practices described in the WOTS framework.

Each CKT matter item was aligned to one of the five specific topics (e.g., changes in matter) and to a specific science teaching practice within one of the seven WOTS instructional tool categories (e.g., scientific models).

The purpose of the score report is to provide meaningful and actionable information to teacher educators about their elementary pre-service teachers’ performance on the CKT assessment about matter and its interactions. To create the teacher educator score report, our team drew upon design principles outlined by Zapata-Rivera et al. (2012), Zenisky and Hambleton (2012), and Zwick et al. (2014), such as providing multiple representations of both individual and class performance and making connections to instructional materials to support pre-service teacher CKT development. We also refined the initial teacher educator score report based on feedback from our project’s advisory board and a convenience sample of seven elementary teacher educators who had extensive experience in science teacher education.

Score Report Components

The teacher educator score report includes five main sections, each on its own tab in an Excel spreadsheet:

• Section 1 (score report overview) describes the purpose of the score report and presents information about the score report content and how to navigate across the different sections of the report.

• Section 2 (scale scores) describes individual pre-service teachers’ scores and aggregated class scores on the CKT assessment about matter and its interactions at the pretest and posttest time points, using various tables and graphs. This section also includes information about measurement error, comparison of the class scores to a reference population, and levels of performance.

• Section 3 (class summary) provides class summaries of the pre-service teachers’ performance by content areas and Work of Teaching Science (WOTS) categories for the pretest and posttest.

• Section 4 (item map) is a tool to help interpret the pre-service teachers’ CKT test scores. The item map ranks a subset of the CKT assessment items by difficulty, provides a summary of the skills assessed by the item, and maps the difficulty level to a scale score.

• Section 5 (acting on your results) provides information about example CKT matter items and instructional packets for use by elementary science teacher educators. This section describes the content areas and WOTS categories included in the CKT assessment and provides links to CKT item examples and CKT packets that can be used to guide instruction based on assessment results.    

This video provides a short overview of the teacher educator score report and illustrates a few of the report’s interactive feature.