The first four semesters of chemistry are known as the “pre-med” sequence but many students aiming for other professional and graduate careers must also take those four semesters and often followed by biochemistry. Each course is a pre-requisite for the next one and only successful students in one course can continue to the next.

The attrition rate through these four courses has remained somewhat constant since 2018 when we first started this new chemistry curriculum. We can attribute the drop in enrollment in the course sequence to different reasons

1. The CHEM1 to CHEM2 drop (around 20 or 30%):

   1. Only CHEM1 is required for graduation. Students who pursue a career that does not require more chem courses will stop in CHEM1

   2. Some students will leave UMR after their first semester either because 1) UMR or college in general is not a good fit for them, 2) they are not academically successful, or 3) they change their major or career other than health sciences

2. The CHEM2 to CHEM3 drop (another 20 to 30% to a total of 50 or 60%). It is for very similar reasons as the first drop but, unlike the first drop, this one cannot be explained by poor academic performance as CHEM1 and CHEM2 are very similar in difficulty. In other words, students who pass CHEM1 should be able to pass CHEM2 (see slide on probability of success)

3. As we can see in the transition tables from CHEM2 to CHEM3 we see a significant portion of students who got A and B in CHEM2 drop the sequence. Therefore these are students who either leave UMR or do not need to continue further.

4. CHEM3 and CHEM4 keep most students. There are some students who skip CHEM3 (Organic Chemistry 2) because there are some careers (e.g. Physical Therapy) that do not require organic chemistry.

 The Chem curriculum can be seen as a story of success as the big majority of students who stay in the sequence will obtain an A or B grade. The most likely outcome is for a student to obtain the same grade that they obtained in their previous chemistry course. The second most likely outcome is one letter grade lower. Except for if a student gets a C the most likely outcome is to drop the sequence, the second most likely grade is a C in the next course.

Being CHEM3 and CHEM4 two very distinct courses (Organic Chemistry 2 and General Chemistry 2) their probability of success is more independent. When going from CHEM2 CHEM4 the most likely outcome is to keep the grade, but the second most likely one is actually to increase it. This may be related to the milestone system graded with “specs grading” implemented in CHEM4 that rewards students who pass them.

The demographics of our incoming cohort of students has remained approximately constant throughout the years. We have had about 30-40% students of color and 20/80 male/female ratio. First generation students are in some years almost 50% of students who report these data.

Everyone needs to take CHEM1331, it is the largest course and the first STEM course required to graduate. It does not easily transfer in, so we can say that the entire cohort needs to take it. Succeeding in this course strongly correlates with staying at UMR. We have developed multiple analysis that help us detect early failure and therefore we can target specific interventions to maximize retention and success in the course and therefore at UMR

Since Fall 2018, CHEM1331 has been a success story. The most common grade is A and B and we have always kept the DF grades at single digit percent. But that does not include withdrawals. Actually, due to our early detection of failure, we can detect most students who are not going to pass by October or November and therefore the number of withdrawals is more indicative of how many students do not pass.

If we look at DFW rates, both Fall 2021 (post-Covid) and Fall 2024 stand out for having larger than usual numbers.

Our numbers show that students who DFW have only a 25% chance to come back to UMR as it is unlikely that they will enroll in CHEM1331 again.

What makes students pass or not pass CHEM1331? We can see in the percentage table that all students who get an A or a B will pass all the milestones (average around 99%). We also see that students who DFW will disengage from all categories and not only exams, including attendance/preclass, homework, labs…etc

Often as educators we think of ourselves as problem solvers and we focus on the problems rather than celebrating and recognizing the aspects of a course that are working well. For example, in these tables we see that the  large majority who pass milestone 1 will pass the course with an A or B.
That being said, milestones can also be used to predict future course failure.
In fact, we can predict that students who do not pass the first milestone will most likely (with a 93.1% chance) fail the course. But that is not all students who DFW, the largest prediction power goes to milestone 2 that is usually the harder milestone. In this latter case, failing milestone 2 gives a 81.7% chance of failing the course.

We have then identified the milestone as the academic assessment that helps us predict success or failure in CHEM1331. It is therefore important that we dedicate special attention to these types of exams. Questions that we have are

1. Multiple attempts: How many do they need? Do they wait until the last attempt? Do they use all the attempts? Do they give up?

2. Do they memorize the questions?

3. If it’s a random pick from the pool of questions. Are they all fair questions?

4. Are there questions that are particularly difficult for some students?

The majority of students will pass on the first attempt. As the semester progresses, there is a slight decrease in the percent of students who get the full passing score both in their first attempt and final attempt. This is understandable if we think that milestones are cumulative so the amount of content and therefore difficulty increases at each milestone.
We see that every first attempt, students who score the “C” about half of them will progress to score the highest “A”. And about one third of the students who fail in their first attempt will obtain the highest “A” in subsequent attempts. We can interpret here that multiple attempts will always benefit a fraction of the students, but not all and in most cases not the majority.

Some students will not attempt further and will settle for a lower grade. The number of “settled” students will increase as the semester progresses to an average of 13% of students in the last milestone not using their attempts to improve their grade. The majority of students settle with a C, but a small fraction will actually give up and not use their attempts even if they had failed in their previous attempts.

The milestone questions are designed to be fundamental and some students may try them over and over again. This is the expected practice. Some concerns exist that some will memorize the questions without true learning. Some preliminary analysis tells us that students do not memorize the answers because when we introduce questions in the real exam that they have not seen before in the practice their performance is not significantly different.

Not all questions are “created equal”. Low-performing students will do poorly in only a subgroup of questions. In other words, the cause for not passing a milestone is not a lower score in all questions, rather, it is three to four questions that have a significantly lower score between passing and non-passing students.