Advancing Cybersecurity with Generative AI: Collaborative Hands-on Learning Modules

The purpose of this site is to host cybertraining materials for Generative AI (GenAI) and Cybersecurity. The goal of this cybertraining is to prepare the future scientific and engineering workforce with advanced skills at the intersection of artificial intelligence and security defense.

Generative AI is one of the most disruptive technologies of our time, capable of creating realistic text, images, code, and simulations. While it offers tremendous opportunities for innovation—such as strengthening anomaly detection systems, generating synthetic training data, and enhancing intrusion detection—it also introduces new cybersecurity risks. Adversaries can exploit GenAI for prompt injection, deepfake phishing, malware generation, and adversarial attacks. Understanding both sides of this dual-use technology is essential for building secure, ethical, and resilient digital systems.

This project seeks to advance workforce development for Generative AI and Cybersecurity through:

1. Promoting AI & Cybersecurity education by developing portable, hands-on labware.

2. Enhancing student active learning with self-contained, real-world case study modules.

3. Disseminating adoptable GenAI labware that bridges the gap between AI innovation and cybersecurity defense.

The project will develop at least five case studies, plus one case study for the Honor program, that demonstrate both attack and defense scenarios:

• Case 1: Prompt-Injection: Exfiltrating “sensitive” data from an LLM

• Case 2: Indirect Prompt-Injection: Exploring attacks where malicious data is embedded in external files

• Case 3: Phishing Email Detection

• Case 4: Scam/Phishing URL Detection: Detects if a URL is a scam/phishing attempt

• Case 5: Code Hallucination & Over-Reliance - Assuming AI output code is correct 

• Case 6: Malware Component Reconstruction Challenge
  
  

Each case study explores a practical application of GenAI in cybersecurity, combining theoretical knowledge with hands-on coding. Every module follows an active engagement cycle:

• Pre-lab activity – Introduces the concepts and key terminology

• Hands-on lab activity – Runs real experiments in Google Colaboratory (CoLab) using Python and open datasets

• Post-lab activity – Analyzes results, reflects on limitations, and explores mitigation strategies

With the open-source Google Colaboratory (CoLab) platform, students only need a web browser to design, develop, and test their experiments. These modules are portable, self-contained, and easily adoptable, allowing students to work remotely at their own pace while accessing interactive lecture slides, code repositories, and discussion forums. Supplemental materials—including instructional videos and lab manuals—will help instructors integrate these modules into related Computer Science, IT, and Cybersecurity courses, particularly to broaden access for underrepresented student groups.

This project will strengthen the cybersecurity and AI curricula, engage students in active learning, and enhance their problem-solving skills for real-world security challenges.

Generative AI & Cybersecurity Modules

Getting Started: Concepts

M1 Prompt-Injection: Exfiltrating “sensitive” data from an LLM

M2 Indirect Prompt-Injection: Exploring attacks where malicious data embedded in external sources

M3 Phishing Email Detection

M4 Scam/Phishing URL Detection: Detects if a URL is a scam/phishing attempt

M5 Code Hallucination & Over-Reliance

M6 Malware Component Reconstruction Challenge

Discussion Forum

If you are interested in joining the discussion group on Generative AI and Cybersecurity, please reach out to the following email address: yshi5@kennesaw.edu to be added as a group member.