10. Integrating ChatGPT in Authentic Peer Marking Assessment  for Process Synthesis and Design in Chemical Engineering 

Authors:  Salman Shahid and Antonis Theodorou

Institution: University of Manchester, UK

Situation 

In the context of chemical engineering education, traditional teaching methods often rely heavily on static problem sets to convey complex concepts and principles. However, these methods often lack engagement and fail to facilitate deep learning and critical thinking. To tackle this, we explored leveraging emerging technologies, such as GenAI, specifically ChatGPT 3.5, to enhance the learning experience for students. This project took place at the University of Manchester in the Chemical Engineering department, emphasising the integration of various chemical engineering principles and GenAI to create efficient and sustainable solutions.

Task 

The project aimed to enhance students’ collaborative design skills by incorporating ChatGPT into chemical process plant design tasks. Students tackled unique problem scenarios using ChatGPT to conceptualise processes, select equipment, ensure safety, and optimise processes for efficiency and sustainability. The project simulated a real-world peer review process, fostering critical evaluation of designs and promoting responsible and conscious use of technology — a crucial aspect in today's technological landscape.

Action 

We introduced ChatGPT into the curriculum, guiding students on its effective use and integrating it into the design process. What made this approach innovative was its comprehensive utilisation of GenAI in the design process while also guiding students on the effective use of GenAI, showcasing its benefits and limitations. Students were provided with comprehensive guidance on the effective use of ChatGPT, offering opportunities for hands-on practice, and encouraging peer collaboration and discussion sessions. Students were tasked with designing a chemical process plant, leveraging ChatGPT at various stages. The methodology spans multiple stages, from problem scenario assignment to draft submission, cross-group evaluation, and iterative refinement. ChatGPT aided in brainstorming, optimising specifications, and assessing safety risks. Cross-group evaluation fostered peer learning and constructive critique, as students were tasked with reviewing and offering feedback on design reports from a different group. Students were provided with training on marking and providing feedback on project reports.

This innovative approach emphasised accessibility and inclusivity via ChatGPT's user-friendly interface, facilitating seamless engagement for all students. Integrating diverse student perspectives and feedback promoted equal participation and fostered a collaborative learning environment. The project scalability allows implementation across different class sizes and settings, with tailored adaptations to meet individual student needs. Additionally, the project equipped students with AI skills tailored for chemical engineering, enhancing their long-term academic and professional success.

Results 

The data indicated that the integration of ChatGPT in teaching has led to improved perceived effectiveness across key areas for students (Figure 1), including Technical Ability, Cognitive Ability, Academic Impact, and Ethical Awareness, with percentages ranging from 65.8% to 76.5%. These results suggest that leveraging ChatGPT in educational interventions has positively impacted students’ skills and awareness in various domains.

Incorporating ChatGPT into teaching design presented several challenges in terms of the accuracy of information, training students to interact with the model through prescribed prompt searches, and integrating it into the workflow despite computational limitations. Despite these challenges, integrating ChatGPT stimulated creativity and improved project quality by optimising designs. However, its best utilisation lies in answering concept-based retrieval questions and providing guidance in more challenging scenarios. Despite occasional complaints about errors, time investment, and comprehension issues, students acknowledged the well-structured process design facilitated by ChatGPT.

The cross-group evaluation, alongside ChatGPT usage, provided valuable insights, aiding students in understanding the limitations of GenAI and gaining diverse perspectives. Iterative refinement based on cross-group feedback boosted students’ confidence in improving their designs, mirroring real-world scenarios where continuous improvement is crucial.

Project findings were shared with university staff through workshops and conferences, offering insights into leveraging technology for teaching and learning. This success may inspire broader integration of AI and collaborative learning methods across disciplines.

Stakeholder Commentary 

Students have recognised both the limitations and achievements of ChatGPT in their academic tasks. However, over 80% of students have expressed positive feedback regarding the integration of ChatGPT into their project workflow. A sample of student feedback is shown in Figure 2. Many have appreciated the ease with which they could brainstorm ideas and optimise design specifications using the model's suggestions, as it helped them refine their project plans and effectively mitigate potential risks. The peer review process has also proven beneficial for students, offering valuable insights into alternative approaches and best practices, enhancing understanding, and improving project deliverables. Furthermore, it has allowed students to pinpoint errors in ChatGPT solution approaches and suggestions, fostering critical evaluation and refining decision-making processes.

Figure 2. Student evaluation of intervention.

Despite the current limitations of GenAI (ChatGPT), it has the potential to enhance education. We anticipate future improvements enabling broader adoption in various teaching practices and fostering dynamic learning experiences in chemical engineering education and beyond.

Author biographies

Salman Shahid 

Salman Shahid is a Lecturer and Director for the postgraduate programmes in Chemical Engineering at the University of Manchester. His scholarly pursuits centre on advancing inclusive teaching methodologies, refining assessment techniques, and enhancing feedback mechanisms.

Antonis Theodorou

Antonis Theodorou is a postgraduate student of Chemical Engineering at the University of Manchester. He is interested in the use of GenAI in chemical engineering education.