See below for an overview/refresher on the project's goals.
Please view this example of what a final assessment item block might look like (it will eventually be in Qualtrics), and this example from math (before they created the selected-response version). (If you're interested, these two papers describe the assessment development from mathematics.)
possible discussion prompts: consider parallels between math and physics -- would you anticipate that an assessment that works to assess math teacher preparation programs would run into snags when thinking about physics teacher preparation? (Spoiler: I have concerns!)
Please take the current survey (link). This will give you a feel for the instrument and its development. (You can decide if you'd like to have your anonymous responses as a part of our study!) Qualtrics will give you two random questions, and you may keep taking the assessment to see more questions. Take as many questions as you like. Please keep notes on questions or issues you run into.
discussion prompts: this is not the final instrument, of course - just collecting responses from physics educators. Do you see any concerns or have ideas for improvement? We will continue to collect data through the summer and can make changes (no statistical work happening at this stage).
(OPTIONAL) We describe the questions in greater detail here. If you'd like to review rationale/thoughts behind the prompts, this is a place to look. (Still being updated.)
Generate ideas regarding "failed" questions and student data. Not all questions will be useful on the final assessment. We can "release" those questions and responses for faculty to use as they see fit. We have been playing around with potential curriculum that use these questions and student responses.
discussion prompts: If you work in training teachers/preservice teachers/LAs/TAs: Would you find the questions + student responses useful on their own to use as you see fit? Would a set of tutorials worksheets be useful, similar to our draft? Do you have other ideas or feedback?
Soliciting participants. For our next stage, we will need a large number of participants to get enough data to have the statistics for the assessment. We would like LAs, TAs, preservice teachers, inservice teachers, and faculty. We can compensate someone who is able to get their program (with a large number of participants) but not compensate individuals.
discussion prompts: Ideas for solicitation of participants?
We will also be sharing information on our current responses and coding, and any questions that emerge for us in the next two weeks!
In this assessment, we operationalize the complex construct of disciplinary attentiveness (or responsiveness) as an educator interpreting the physics goals of a task, analyzing student work in light of those goals, and responding to student ideas in ways that are likely to move those ideas towards disciplinary norms. The assessment is designed to be used as an assessment of a program, not an individual.
We have the following steps in developing the assessment:
1 - Developing a suite of questions for secondary students: these are questions that are designed to elicit common (but canonically incorrect) ideas, and should represent the kinds of questions and activities that an introductory physics or physical science teacher would ask. A challenge for us is the range of ideas that might be elicited by a single question.
2 - Creating a free response survey for physics educators (TAs, LAs, preservice and inservice teachers): we select student responses from step 1, and use those to solicit responses from educators. Their responses will be used to generate authentic selected-response items in step 3. We ask physics educators to look at the task and a student response, and describe the intent of the task, describe the student's reasoning, and describe what next steps they would take as an educator.
3 - Creating a selected-response survey for physics educators: we will code responses (see here) and select representative responses for the selected-response assessment. These will be administered in 2023-2024 and our statistician will support the analysis.