Teaching-as-Research

Many of the labs in College Chemistry I at Madison College have limited connection to things that students encounter outside of chemistry class. Most are focused on “identify the unknown” type problems that are not representative of what scientists actually do. My Delta Program internship was focused on replacing a previous lab in the course with one that incorporates more authentic scientific investigation and local environmental samples, and evaluating the impact of the new lab. Specifically, I wanted to test how the new lab impacted student engagement. This section of my website is specifically focused on how I evaluated the new lab and the other labs in College Chemistry 1. To read about the lab itself, see this page. For relevant literature review on student engagement and the use of environmental science in chemistry labs, see this page.

Research question: Will a lab with applied, real world applications increase student engagement and understanding in College Chemistry I at Madison College?

Hypothesis: Student engagement will be higher in the lab with real world applications relative to labs with limited connection to the students.

Evaluation Methods:

To evaluate student engagement with the new lab relative to other labs in the course, I used a short survey, which the students filled out every time they turned in a lab report. The survey had five questions: four multiple choice, and one open ended. The full text of the short survey is below:

1. I understand how the knowledge I gained in this lab may be used outside of chemistry class.

Strongly Agree Agree Neutral Disagree Strongly Disagree

2. This lab (as a whole) was interesting to me.

Strongly Agree Agree Neutral Disagree Strongly Disagree

3. These components of the lab were especially interesting to me (circle as many as apply):

a. Pre-Lab activity

b. Pre-Lab discussion

c. Performing the procedure

d. Performing the data analysis

e. Post-Lab questions

f. Other (please specify):

4. These components of the lab were especially helpful for my learning (circle as many as apply):

a. Pre-Lab assignment

b. Pre-Lab discussion (in lab)

c. Performing the experiment

d. Data analysis

e. Post-Lab questions

f. Other (please specify):

5. Write a one sentence summary of what you learned in this lab.

Students were assigned an alpha-numeric identifier at the beginning of the semester to avoid association between their names and survey responses. Alpha-numeric identifiers were written at the top of each survey.

In addition to the surveys about each lab report, a longer survey was given at the end of the semester. This included 7 open ended and 4 multiple choice questions about students’ intended major, planned future chemistry coursework, expected grade in the course, favorite lab exercise, memorable parts of labs, recommendations for improving the lab portion of the course, and overall impressions of the lab. The full text of the survey is available here.

Weekly Survey Results

Student responses to the first two questions are summarized in Figure 1. We assigned a number to each level of the Likert scale (from one for strongly disagree to five for strongly agree), and averaged the results from all surveys for each lab. As the graph demonstrates, there was only small variations in student responses across all of the labs, with the majority of students responding “Agree” (4) for each question on all of the surveys.

Figure 1: Student responses to the first two questions on the short surveys given after each lab report was turned in. There were minimal differences in responses between labs.

According to question three on the short survey, students found performing the experiment to be the most interesting component of seven out of the nine labs assessed. Data analysis was the most interesting component for the remaining labs, including the new lab on chloride measurement. The responses to question four show that data analysis and post lab questions were the components most helpful for learning in all but one lab. Students found that performing the experiments was the most helpful component for the Mystery Solutions exercise. Table 2 shows the number of students that circled each answer in questions three and four for each lab. Responses to questions three and four on the survey regarding the chloride lab are shown in Figure 2.

Table 2: The number of students who listed each response for questions 3 (interesting components, blue) and 4 (components helpful for learning, orange) on the weekly lab report surveys. The highest responses for each lab are shown in bold, with darker fill color.

Figure 2: Results for questions 3 and 4 on the short survey about the new lab – Analysis of Chloride in Local Water Samples.

There were a variety of student responses for question 5 on the chloride measurement lab survey. Responses could be sorted into nine categories. Table 3 shows a summary of student responses. In general, the responses to question 5 (write a one sentence summary of what you learned in this lab) were fun to read, but provided little insight into student engagement with the labs. Thus, a full qualitative analysis of student responses is not provided in this report.

Table 3: Responses to question 5 regarding the new chloride lab

Final Survey Results

All students expected to earn at least a B in College Chemistry 1, and feedback on the course and instructor was very positive. Commonly mentioned “memorable experiences” include using the spectrophotometer (5 students mentioned specifically) and learning to use Excel (3 students mentioned specifically). 17 students wrote that nothing in the course needed to be changed and there were many positive comments such as “It’s a great class”, “Christen is doing everything so well there is nothing I can add”, and “Everything was very interesting/practical and related to lecture”. Criticisms included uncomfortable chairs in the lab, a bad projector, and the need for more emphasis on calculations and sig figs in class.

The majority of the 32 students who filled out the final survey fit in to one of two career groups. 18 students have plans to go into the medical field (self-reported majors include nursing, physician’s assistant, biomedical engineering, medical school, medicine, pharmacy school, and radiation therapy) while 13 plan to enter science and engineering fields (self-reported majors include biochemistry, food science, biology, mechanical engineering, industrial engineering, and undecided engineering). One student plans to go into business. The students can also be broken into groups based on whether or not they plan to take additional chemistry classes, with 11 students planning to stop after College Chemistry 1 and 21 planning to continue at least through College Chemistry 2. These groupings serve as a useful way to further break down the other data to compare between groups of students.

The responses to the Likert scale (1 indicates not at all, 10 indicates very much so) questions on the final survey were largely positive, with average ratings of 8.3/10 for “Lab helped me understand chemistry concepts”, 8.0/10 for “In lab, I learned how chemistry can be used in the real world” and 7.7/10 for “Lab was interesting and fun”. The average rating for “My experience in lab this semester was different than my expectations for the course” was 5.6/10. Average rankings were higher for the students planning to take more chemistry classes and for students planning to go into science/engineering fields

26 students listed their favorite lab, with some listing more than one (all listed were included in the analysis). The most popular lab overall was the Mystery Solutions exercise, with 8 students identifying it as their favorite. Spectroscopy of Wine and Double Displacement Reactions were also popular, with 5 responses each. Measurement of Chloride was identified as a favorite lab by 4 students. Analysis of subsets of students is difficult due to the small number of responses and large number of choices; the majority of labs appear to be equally popular across groups. However, all 4 students who listed Measurement of Chloride as a favorite lab are in the medical field, and 3 do not plan to take more chemistry classes.

The groupings of student can also be used to break down the weekly survey data. Figures 3 a and b show student responses to the first two weekly survey questions separated by career path and future plans. While responses showed only small variations, some trends can be deduced. For 8/9 labs, the science and engineering students understood more about how the lab applied to the world outside of chemistry class, while the medical career students found 7/9 labs more interesting. Students who plan to take more chemistry classes found all of the labs more interesting and understood more outside applications than students who do not plan to take more chemistry.

Figure 3 Weekly survey results separated by student career/major plans (a) and future chemistry course plans (b)

Discussion

The weekly lab surveys were intended to provide a basis for comparing the different lab exercises. However, the results were very similar between labs and do not provide much insight into how student engagement varied between labs. When the weekly survey results are separated based on the self-reported information from the final survey, average responses are still all within 1.5 units on the Likert scale. The lab that stands out the most in these comparisons is the Spectroscopy of Copper Sulfate, which has the largest response gap between the students who plan to take more chemistry and those who don’t and between the students pursuing a degree in the medical field and those pursuing science/engineering degrees. Future improvements to the College Chemistry 1 curriculum may include finding a way to make this lab more engaging for students who are pursuing medical careers and students who do not plan to take more chemistry. However, the Copper Sulfate lab also received a lot of positive feedback on the final surveys, as five students listed using the spectrophotometer (used in both the Copper Sulfate lab and the Wine lab) as a memorable experience.

The weekly survey specifically about the Chloride Measurement lab shows that students found the data analysis was the most interesting component of the lab and the component that most helped their learning. This result makes sense, as the data analysis both required the use of molarity calculations that were a concurrent topic in the lecture class, and revealed the results of the experiment. For most of the other labs, performing the procedure was identified as the most interesting component, but titrations (while an important staple of many general chemistry lab courses) are not very exciting to perform, and due to the use of combined class data for analysis, students could not see any environmentally relevant results while completing the procedure. Relative to the other labs, the Chloride Measurement pre-lab discussion was identified as more interesting. This could indicate increased engagement in this lab itself, or could be because there was a guest instructor teaching the lab. The majority of the one sentence summaries of what was learned in the lab focused on environmental applications such as water quality. This indicates that many students cared about the implications of what was learned in the lab, beyond how to complete a titration. However, the lab report scores show that the students adequately learned the chemistry concepts as well.

The chloride lab was listed as a favorite lab by four students on the final survey. None of the four were science/engineering majors, and only one plans to take future chemistry classes. While this is a small sample size, the Chloride Measurement lab appears to be especially popular amongst the groups of students who generally have less interest in chemistry (according to the final survey). The Mystery Solutions lab, listed most frequently as a favorite, is unique in that students develop their own procedure for the lab. While it would be difficult to have students developing methods early in the semester, future course updates could include redesigning some of the other labs to have more student involvement in deciding on procedure details.

While it is difficult to make comparisons between labs using the data collected, the new lab on Chloride Measurement seems to adequately engage students in learning about both environmental and chemistry concepts. In that regard, it should be a considered a success.

(next page: Reflection)