Physics Case Study

Course Background

The PH 121-122 General Physics course is a service course offered by the Department of Physics. There are some unusual properties of the course and the audience that lead to some interesting challenges and interesting opportunities:

  • The students in the course are generally life science majors—biology, health sciences, and related majors. In the second semester course, nearly 100% of the students in the class are life science majors.
    • Students in the course understand the nature of science, and find the natural world an interesting topic of study.
    • In order to fully engage with the course, students need to understand why this course is relevant to their majors.
  • The course is required for their majors, but it is not a prerequisite for their upper level classes, and the differential tuition structure provides a financial incentive to structure enrollment so that it is taken later in students' academic careers. Though the course is at the 100 level, most of the students in the class are juniors and seniors.
    • Rather than being a foundational course that serves as a basis for understanding other sciences, the course serves as more of a capstone course to help integrate and explain content from other courses.
    • Students in the course must complete it successfully to avoid disruption to their academic careers; the incentive to pass with a C or better is significant.

One instructor in the Physics Department has taught this course for nearly 20 years. He serves as an informal course coordinator, providing advice and assistance to other non-tenure track (NTT) faculty who are teaching sections of the course. He also is co-author on the textbook used for the course. This unusual situation makes certain aspects of this course unique:

  • The course has evolved to include accepted best practices from Physics Education Research as well as other areas. Changes in recent years have been small; the course as offered is rather mature.
    • Changes to the course will be evolutionary rather than revolutionary. Innovations are more apt to succeed, given the thoughtful design of the course, but dramatic changes in outcomes are unlikely.
  • The lead instructor's participation in creating a textbook has given him an excellent window into the content and the level of the course at institutions around the country.
    • The instructor is able to gauge precisely the cognitive difficulty of different tasks, different problems. One of the changes to the course in recent years is an increase in rigor, as judged by this experienced instructor. Students have been presented with more challenging problems on homework and on exams.
  • In recent years, other NTT faculty who are teaching the course have adopted the textbook and many of the lead instructor's instructional materials. The course is reasonably consistent from semester to semester, instructor to instructor.
    • Changes in this section of the course are likely to be adopted in other sections as well, making this course a good place to start for introducing changes in this particular sequence.
    • There is more variability when tenure track faculty are teaching; there is no formal agreement of how the course should be taught, and they are more likely to use different texts and approaches.

Redesign process

Over the past two years, the lecture and online sections of the course have been extensively revised, with the advice and assistance of the instructional designers at TILT. The design was also informed through discussions with other instructors taking part in the APLU grant.

There are two crucial changes in the course:

  • Increased level of active learning in lecture sections. Lecture sections in the course have always included interactive components; students have worked in groups to answer clicker questions and other short exercises. In Fall, 2016, the course was moved to Johnson Hall 222. This room has tables with whiteboard tabletops that can be grouped in pairs. Now, students in lecture sections work in small groups approximately half the time, guided and assisted by a group of learning assistants. Key features:
    • The learning assistants have recently completed the course themselves. There is good evidence that such "near peer" instruction is most effective.
    • Students work on problems similar to those that they will see on homework and exams. Students see, immediately, the connection of lecture work to assessed elements of the course.
    • The instructor meets with the learning assistants each week for 1-2 hours, going over the material, discussing issues that have come up in class, and giving advice about effective questioning strategies and other topics. Some of the learning assistants also take part in a formal training program through TILT.
    • The problems that the students work on are carefully scaffolded, with enough steps broken out to provide guidance as students learn problem-solving skills.
  • Integration of adaptive courseware. MasteringPhysics, the adaptive courseware connected with the textbook, is used for reading quizzes and mastery exercises. These online components bookend the weeks:
    • Students read a chapter in the textbook, using online prelecture videos if desired.
    • Students take a reading quiz on the chapter, and are given immediate feedback on topics that they didn't quite grasp so that they can take another pass through this material.
    • Students come to class ready to work, to begin learning to analyze physical situations and solve problems.
    • At the conclusion of the week, students complete an online mastery exercise to assess their level of expertise. If they struggle with problems on this exercise the software assigns similar exercises with embedded hints to help them master the concepts and techniques of the chapter.

Results

We are continuing to assess the results of the changes that we have made. In Fall 2018, we will assess the students in the course to compare their content fluency and self-efficacy vs. students in similar courses around the country using standardized instruments from the Physics Education Research community.

To date, we have been able to demonstrate rather striking results on student success.

Course Participation

The social nature of in-class interactions has increased the level of engagement with the class. There are several quantitative measures:

  • Daily class attendance has increased from an average of approximately 80% to nearly 90%.
  • Students are more likely to complete their homework, less likely to have excused absences for lab and recitation.

This increase in student engagement has also led to an increase in demands on the instructor and the learning assistants. The collaborative nature of the instruction means that students are more comfortable asking questions.

  • Rather than hold traditional office hours, the instructor supervises problem sessions in the regular classroom where students gather to work together on assignments. Approximately 1/3 of the students in the class come to an office hour / problem session each week. This has led to a dramatic increase in demand for learning assistants that has created a welcome problem—trying to figure out how to serve the needs of students who are working hard to learn the material of the course.
  • The instructor receives dozens of e-mail and Canvas messages from students in the class on a typical day, many more than in the past. Again, this has created a welcome problem—trying to figure out how to keep up with questions from students.

Learning Gains

The Spring 2016 course was the last semester taught in the traditional format; the changes noted above were put in place in Spring 2017, and further refined in Spring 2018. We will compare results of these three semesters below. As noted above, the course has been in development for many years, and was in pretty good shape. Nonetheless, we were able to see some definite increases in student learning.

Outcome #1: The course is more effective at teaching students how to use physics to reason about the world around them.

  • In the past, students would often give numerical answers that were comically large or small; we are much less likely to see this now. Students tend to give answers that mesh with reality. Numbers might be off by a factor of 2, but they are less likely to be off by a factor of 2 million.
  • Students show more ability to reason qualitatively. The reading quiz questions are qualitative in nature, asking students to reason about physical situations without using numbers. Comparing Spring 2016 to Spring 2017, the percentage of students who got more than half the questions correct on the weekly reading quizzes increased from 91% to 94%.

Outcome #2: Students in the course have shown definite increases in their ability to solve the types of problems we give on tests.

  • The first tests in the Spring 2016, Spring 2017 and Spring 2018 had similar levels of difficulty, but student performance varied considerably:
    • Exam #1 grade, Spring 2016: 60%
    • Exam #1 grade, Spring 2017: 67%
    • Exam #1 grade, Spring 2018: 71%

Outcome #3: In response to increased student performance, the instructor was able to increase the level of rigor in the course in subsequent sections; final course averages in the different years showed a slight decrease:

  • Average course grade, Spring 2016: 84%
  • Average course grade, Spring 2017: 83%
  • Average course grade, Spring 2018: 82%

Despite the increased level of rigor, the DFW rate in the course stayed very low:

  • DFW rate, Spring 2016: 3%
  • DFW rate, Spring 2017: 2%
  • DFW rate, Spring 2018: 2%

Increasing rigor with slightly lower course averages but slightly lower DFW rates reflects an interesting change in the course grades—the distribution of grades lost its long tail in the lower grades, and became more symmetrical.

  • In Spring 2016, the median grade was 2.4% higher than the mean grade; there was a small, but noticeable, tail in the lower grade ranges.
  • In Spring 2017, the median grade was 0.9% higher than the mean grade; this means the distribution was nearly symmetrical, lacking the long tail at the lower end of the grade range.

In 2018-2019, we will do a more careful assessment of learning gains to see if these results continue to hold.

Challenges

There are 4 primary challenges to continuing these innovations and extending them to other courses:

  1. Support
    • The course brings in a great deal of revenue (the regular sequence of General Physics averages 500 students per semester, nearly all of whom are subject to differential tuition surcharges) but does not receive a corresponding level of resources.
      • The current budget is not sufficient to provide learning assistants at the necessary level. If we are to continue using learning assistants, and to extend their use to other courses, we will require additional resources.
    • Supervising learning assistants and dealing with the increased level of student engagement has been an additional burden on the instructor, and there is not support for reducing effort elsewhere or giving additional assistance.
      • Moving to this new model of instruction has meant, for the instructor, approximately 5 extra hours per week, in meetings with learning assistants, in increased correspondence with students, and increased numbers of hours for problem sessions. All of these are desirable changes—who can argue against increased contact with students?—but the fact that this new model means spending more, rather than less, time teaching makes it a hard sell to other instructors.
  2. The nature of the course affected
    • The course described is a service course that is a relatively low priority for the Department. This means that changes are less likely to trigger reactions, for good and bad. No one is likely to complain when we change the way the General Physics course is taught, but successes here are less likely to spur people to try a similar approach in another course.
  3. The nature of involved faculty
    • The course is taught, as a rule, by NTT faculty. These instructors have instruction as the heart of their priorities, and are judged by the success of their instruction. They are more open to trying new approaches, but these approaches are less likely to be adopted by tenure track faculty because they take more effort and because, for these faculty, professional success and advancement is only weakly connected to their performance in the classroom.
    • NTT faculty are less likely to stay in their positions for a long time, and the changes they have implemented will not always survive their departure.

4. Class size

    • The learning assistants do an excellent job, but contact with a caring and committed instructor who is well versed in the material and modes of instruction is even better. Unfortunately, the large size of the General Physics classes (lecture sections range from 200 - 280 students) means that most students have interactions with the course instructor only rarely. Smaller class sizes would certainly increase engagement and effectiveness.

Final Words

It has been very rewarding to work with the staff at TILT on this instructional redesign. We have been able to increase engagement, and to increase rigor while also reducing student failure rates from an already very low level.

We have a model that works, and it is acknowledged to work within the Department. We have gone about as far as we can in the current situation. Extending this model to other Department courses will require some institutional changes in funding and reward structure.