Impact of Supplemental Instruction Sessions on the students: A case study


Sushma Anand Akoju

Background

Foundational theory courses in Computer Science are mandatory courses for the students enrolled in Computer Science Undergraduate and Graduate programs. One such foundational Undergraduate course is CSC 473 Automata, Grammars and Languages. This is a course that is theory-intense, requiring understanding of types of languages, syntactic & semantic properties of languages, theory of computation and complexity of the computational methods that recognize and/or decide a language. 

Among such foundational theory courses in Computer Science, the solutions are standardized to the problems that test the comprehension of the students. Hence there is a limited scope to the variations in the already standardized solutions that are experts' agreed. However such a restriction is often challenged by the solutions that exist on the Internet with/without sufficient justification and alignment with the steps in solving the problem to match the available solution. The votes given by the internet users often represent such solutions as more reliable and seemingly correct. [28]

Given a correct solution, the goal of finding the steps to solve a problem such that it matches an available solution can become a more stressful goal to a learner. Such a goal depends on the complexity of the topic and/or homework problem itself and the comprehension of the concepts.

During the Fall semester 2023 for CSC 473 class, students shared their challenges that they faced such as increased stress levels, spending more time on homework assignments due to external factors and lack of Computer science background. [16] Students came up with questions such as how to align an internet solution back to the lecture materials. There are several internet resources such as StackOverflow, and paid resources such as Chegg, Studocu etc and AI tools such as ChatGPT, that impact how the homework assignments are solved. [4][7][8][12][13] [22][23][25][28]

When the learner has access to the correct solution before the learner has solved a homework assignment problem, then that might inhibit the chances to fully engage in - applying their knowledge & comprehension of lecture materials and learning from their mistakes.[1] This could have positive as well as negative effects. Positive effects include showing good comprehension of the learner where the learner is able to explain how and why the solution from the internet is indeed correct. Negative effects include failing to demonstrate comprehension of lecture materials/concepts, due to missing steps or insufficient justification of the available internet answer etc. [22][23][25] 

The internet sources such as StackOverflow are Community-based Question Answering (CQA) websites.[28] The CQA website answers are user-voted and provide topic-based tags. But do not provide information if a question is posted for an undergraduate or graduate level course since course modules, learning goals and outcomes differ between the two types of courses. Besides, we generally expect to share a solution with an expert rather than a learner for evaluation if the answer was correct.

In addition to these challenges, internet has multiple methods for solving a problem. Some students shared a feedback that they spent additional 3 to 4 hours finding the "right" or "easy" method to solve the homework assignment problem that caused more stress to the students. Besides, methods found on the internet may or may not align with that of the methods discussed in the lecture materials. There are several open-ended questions to assess an individual learning process in this specific scenario of applying a method found on the internet to solve a homework problem. 

Undergraduate students, seem to report certain concerns w.r.t pace, increased anxiety levels from some of earlier studies, for introductory courses during the semester.

Adding to these problems, is the need for emphasizing when to focus on individual learning versus competitive and/or collaborative learning. [19][20] Learners are under constant pressure from competitive environments but to not only do well in the subject but also do better than their peers.

The aforementioned challenges, emphasize one thing with certainty based on students' feedback: there is an increase in stress level as well as longer amounts of time spent in solving homework problems.

To address these problems, I was keen to help students with Supplemental Instruction sessions such that the sessions are part of the syllabus and were more structured so they get a chance to apply the information learned from the lectures. With the additional guided & collaborative practice sessions, I wanted to help - improve the comprehension of the students, reduce stress levels and reduce amount of time spent on homework assignments.

I, as a Teaching Assistant for CSC 473, worked on weekly practice sessions preparation and execution of sessions, to study if the practice sessions helped the students reduce stress and amount of time spent in solving homework assignments. This project is part of a course "Teaching as a Research", IA 699, from Center for the Integration of Research, Teaching, and Learning (CIRTL), University of Arizona.

Introduction

Evidence-based teaching and Learning [2][3]

"How does this solution align with class lectures?"

Image Credits: Bloom's taxonomy revisited 

Challenges from solutions available on the Internet and AI tools as Situational Factors

What is a Supplemental Instruction Session?

A Supplemental Instruction (SI) session is a free out-of-class session that is designed to help the students learn how to apply the information learned in the class. [14] SI sessions are additional practice, learning and/or revisions sessions. Commonly, SI Sessions help with additional practice with problem solving, revision of the lecture topics, preparation for tests/exams and answering the students' questions. [18] University of Arizona provides free access to Supplemental Instruction (SI) sessions that are led by Teaching Assistants or trained students.

The Teaching-as-a-research (TAR) project is to study if the SI sessions help in the reduction of stress and reduction of amount of time spent on homework assignments in the course CSC 473 Automata Grammars and Languages, Spring 2024.

The SI sessions were optional in Fall semester 2023 and for Spring 2024, more changes in the format of the SI sessions were introduced. I worked as a Teaching Assistant (TA) for the same course, during Fall 2023. Based on the feedback received from Fall semester, the SI sessions were made part of syllabus, although optional for Spring semester 2024

Teaching As a Research (TAR) Project

During the Spring semester, 2024, we introduced few changes to the SI session format, which attempts to address concerns regarding students' high stress levels and longer duration of time spent on the homework assignments. I shared the feedback I received from the students from my Office hours as well as SI sessions from Fall 2023, with the professor. 

Prof. Cesim Erten provided changes towards the design of the SI sessions and included them as part of the class syllabus during Spring 2024. Prof. Cesim Erten taught CS 473 during Fall 2023 & Spring 2024 semesters during which I worked as a Teaching Assistant. I worked with professor under guided supervision for the preparation and execution of the SI sessions.

We used the interactive classroom activities with learners and the instructors likewise. Based on the situational factors and best practices from Bloom’s taxonomy, there is an emphasis to practice by first understanding before remembering the method & steps to solve the problem. The theoretical subjects are generally not easy to self-verify if a concept was well-understood or not. Thus TAR project is designed to combine the concepts learned in the IA 699 course with that of design and application of SI sessions that align with CSC 473 course requirements.

We expect the SI/practice sessions would address the challenges from Fall semester 2023, since they are guided and collaborative in nature. [5][6] We ask the students to exchange the solutions between the peers during the SI sessions. We provide corrections, additional explanations and revisions as needed. The SI sessions could help to reduce both the stress levels and amount of time spent solving homework assignments of the learners. [14][16][18]

Changes in design of the SI sessions for Spring 2024, CSC 473  for the TAR project

During Spring 2024, the goal of the SI sessions was to verify if those students who did attend SI Sessions, have:

We ask all students to enter this information for each one of Homework assignments 3, 4 & 5.

We compare the two groups: students who did attend SI sessions versus  students who did not. The TAR question is described in following section. 

TAR Question

Do students who attend SI sessions, of CSC 473 course, demonstrate significant improvement in time spent and reduction in stress when solving homework assignments 3, 4 & 5 compared to those who didn't attend?

Learning Goals and Outcomes

Based on the feedback and surveys from the Fall 2023, we streamlined Supplemental Instruction (SI) sessions as Practice/Discussion sessions for Spring 2024. The design, weekly preparation and execution of the SI sessions are supervised by the Professor.

Learning goals:

To provide additional practice for the students with the help of SI sessions. 

Learning Outcomes:

The students spent less time and experienced reduced stress levels in solving a problem due to the SI sessions.

Type of Assessment:

Formative assessment, since there is no grade or reward for merely attending practice sessions. However, there are other summative assessments included in the syllabus but is out of scope of this project.

Approach / Methods

The modified SI session design and activities are:

       Figure 1: Weekly SI session preparation Cycle, Spring 2024, CSC 473 

About the Data Analysis Plan and the Results

The learning outcome assessed for TAR question is:

Students who attend the SI sessions will:

Data collection:

Data Analysis changes:

We grouped stress ratings of 1 to 5 into 3 Stress levels as follows:

Qualitative Analysis Results

Data Visualization and Data Analysis Results for Homework assignments 

3,4 & 5

Figure 2: Percentage of students attending SI sessions for Homework assignments 3,4 & 5

We note an increase in percentage of students attending SI sessions for Homework assignment 3, but for 4 and 5 there is relatively lower percentage of participation in the SI sessions. 

We also note the period after mid-semester shows a certain trend of other factors, such as midterms, mid-semester project works in other courses and so on.

Figure 3: Stress levels across Homework assignments 3,4 & 5

Stress levels vary among three homework assignments between the students who did attend SI sessions and who did not.

There are high, medium and low stress levels. Among each group of students who attended SI session & did not attend SI session, the percentage of the students with each one of three stress levels are marked.

As this homework assignment 4 also includes spring break, the stress levels are higher for the students who did not attend the SI session due to the external factors such as going on a vacation etc. This contradicts the assumption that "spring break might lead to reduced stress levels and less time spent on homework assignments".

Although homework assignment 5 is closer to the 2nd midterm exam, the stress levels are moderate among the students who attended the SI session.

Figure 4: Total time spent in solving each one of the Homework assignments 3,4 & 5

For each homework assignment, the percentage of students and the amount of time spent in hours for solving each one of the 3 homework assignments, among the students who attended SI sessions versus who did not, are marked

This plot shows that attending SI session may be associated with that of the amount of time spent on homework. Students who attended SI sessions spent relatively lower amount of time in comparison to students who did not attend the SI sessions.

And homework assignment 4 also includes Spring break, there maybe influence of the external factors such as going on a vacation etc. Contrary to the assumption that spring break etc could reduce amount of time spent on homework assignments, the data shows that students who attended SI sessions, spent more time during the spring break solving homework assignment 4. But students who did not attend SI sessions, spent relatively less time in solving homework assignment 4 but experienced higher stress levels as seen in Figure 3.

Discussion & lessons learned from the TAR project

What worked well? 

Known challenges before conducting the study:

What challenges did you have during and after conducting the study? 

What surprised you?

What did you learn by doing TAR? 

I learnt that teaching as a research is an ongoing learning goal that needs to be flexible.

How has this project informed how you will teach in the future? 

What do you want to continue learning? What excites you?

What have you learned as an educator?

Overall it was an enriching experience working on this TAR project. As a teacher, we have many chances to learn from these experiences, in general. But with TAR project, the teacher's learning also can be more structured and aligned with that of CIRTL, department policies, university's core values and teaching methods.

About the Author

Sushma Akoju, is a PhD Student. My research experience details are available at: orcid

Summary of my TA experience for CSC 473. 

I believe teaching and learning experiences are best when learners and teachers are engaging in open conversations and when we work with each other to meet the learning goals.

References:


Special Acknowledgements


CIRTL & TAR project Instructors Kristin Winet and Byron Richard Hempel:

It was fun to be part of an enriching and supportive learning community such as CIRTL, University of Arizona. I am truly grateful to Kristin Winet and Byron Richard Hempel, for the support, guidance and learning experiences. Thanks to CIRTL!!

Professor CSC 473, Automata, Grammars and Languages, Prof. Cesim Erten:

The TAR project is small subset of overall course design and syllabus of CSC 473, Automata, Grammars and Languages. Although I actively participated during Office hours, answering students' questions while conducting Teaching Assistant (TA) duties, this project is heavily based on overall assessment, discussion with the Prof. Cesim Erten. I am grateful for the professor's support, guidance, encouragement to work on this project as a Teaching Assistant. 


Students of CSC 473, Spring 2024 & Fall 2023:

Special thanks to all the students of CSC 473 during Spring 2024 as well as during Fall 2023.

I am truly grateful to receive active participation from the students of Spring 2024 semester and taking time to answer the questions related to this study for the homework assignments (3,4 & 5). 


Professor CSC 573, Theory of Computation, Prof. Roberto Giacobazzi:

I was also simultaneously enrolled into CSC 573 Theory of Computation course taught by Prof. Roberto Giacobazzi, University of Arizona. The subject deep dived into foundational concepts required for Graduate level course. Prof. Cesim Ertem and Prof. Roberto Giacobazzi also aligned concepts between both of the classes together. Together with an access to advanced, expert-guided learning, I utilized the opportunity to seek a deeper understanding of the subject. I am truly grateful for the chances to be enrolled in CSC 573, while being a TA for CSC 473 and working on the TAR project.


Special Thanks to the Writing Center, University of Arizona (ThinkTank):

Thanks to Writing Center for proof reading this website. 


TA Osho Sharma:

Thanks to Osho for assisting me with some parts of the SI sessions during Spring 2024.


TAR classmates:

Thanks to the classmates for the support and it was fun learning experience.

Recommendation

I recommend CSC 473 Automata, Grammars and Languages course, University of Arizona, for future Undergrad students, as it is a very helpful course with great resources and tools to help students do well. 

I recommend CSC 573 Theory of Computation course, University of Arizona, for Graduate students, for the enriching learning experience. 

I recommend IA 699 Teaching As a Research course for any Teaching Assistants interested to pursue this amazing course for learning about teaching as a research.