This page might be updated without prior notice. Please look out for notifications on Canvas and check this page frequently.

Last updated on 08/20/2026

Announcement

[Update on 09/21] Due to a conference travel, the office hour on Wed 09/23 and the lecture on 09/24 need to be cancelled. We will resume our lecture on 09/29.

[Update on 09/03] Attendance link: https://docs.google.com/spreadsheets/d/11kKZzwvFGGtKnidvzIT5-Nn_GKj0BKdOmL81A3rnuWQ/edit?usp=sharing

[Update on 08/20/2026] Presentation schedule has been updated to reflect on the academic calendar.

[Update on 08/10/2026] Welcome to our first of its kind AI system design course at UNC Charlotte!

Course Information

• Lectures: 17:30 pm - 18:45 pm Tues/Thur, CHHS 380

• Office hours: 4pm - 5pm Wed or by appointment

• Instructor: Jinzhen Wang, jwang96@charlotte.edu

• Canvas: https://instructure.charlotte.edu/courses/270998/

• Course website: https://sites.google.com/charlotte.edu/ai-system

• Slack channel: https://join.slack.com/t/aisystemdesig-g9z5783/shared_invite/zt-471599k86-sc8sPgPa_kEqXK2WxmiJAg

• Presentation schedule: https://docs.google.com/spreadsheets/d/1u3A-WiEfwWS-wZRnxf-cOYyvL8G_Ja6a_inW1cVYtFc/edit?usp=sharing

• Syllabus: https://drive.google.com/open?id=13qjZOHIQY3pzoIueJohs8pcFh7SQMtY3&usp=drive_fs

• Feedback form (anonymous): https://forms.gle/PKojDp9PyjfSujxN7 (anything from anyone is welcome)

Course Description

Artificial intelligence systems increasingly underpin modern software infrastructure, yet designing reliable, scalable AI systems requires knowledge that spans distributed systems, machine learning, hardware, and data engineering. This course introduces the principles and architecture of end-to-end AI systems, with explicit focuses on:

• Traditional non-AI software systems

• Traditional AI / machine learning systems

• Edge AI systems

• Large Language Model (LLM) systems

Students will learn how to design systems that satisfy performance, reliability, scalability, and cost constraints. The course also introduces emerging HPC × AI workflows, where large-scale scientific computing infrastructures are integrated with AI pipelines.

Learning Objectives

The course emphasizes system architecture, trade-off analysis, and end-to-end design rather than isolated algorithms. By the end of the course, you will be able to:

• Analyze trade-offs in large-scale system design (latency, throughput, reliability, cost).

• Design scalable non-AI distributed services.

• Build and evaluate machine learning pipelines and production ML systems.

• Optimize AI models for resource-constrained edge devices.

• Architect and evaluate LLM-based systems including retrieval-augmented systems and agent frameworks.

• Integrate AI workloads with HPC infrastructure and scientific computing pipelines

Prerequisites

There is no prerequisite to take the course. However, you need the following skill sets and knowledge to thrive.

• Intermediate familiarity with Machine Learning and Deep Learning

• Intermediate familiarity with Operating Systems or Distributed Systems

• Basic proficiency of Programming in Python or C++.

Learning Resources

Books

• Designing Machine Learning Systems, Chip Huyen

• Designing Data-Intensive Applications, Martin Kleppmann

• AI Systems Performance Engineering, Chris Fregly *

• The System Design Primer

External materials

• CS 329S: Machine Learning Systems Design

• CS294: AI Systems (LLM Edition) 

• CS294-162 Machine Learning Systems 

• 15-442/15-642: Machine Learning Systems

Course Activities

Weekly lectures consist of 

1. regular lectures on specific topics (50%, by the lecturer, on Tues); 

2. presentation on a case study (25%, by one student, on Thurs); 

3. panel discussion (25%, by all other students, on Thurs);

All students are required to submit a literature review for each module as homework assignments, usually dues 2 weeks after the end of each module.

The overall grading will be based on presentation (34%), panel discussion (36%), and homework assignments (30%).

Lecture Outlines (tentative)

• Module 1 - System Design Foundations

  • Lecture 1: Introduction to AI system (Logistics)

  • Lecture 2: AI System Overview

• Module 2 - Non-AI Systems

  • Lecture 3: Scaling under load

  • Lecture 4 - Designing for failure

  • Lecture 5 - The data layer

• Module 3 - Traditional AI Systems

  • Lecture 6 - 

  • Lecture 7 - 

  • Lecture 8 - 

  • Lecture 9 - 

    • Compression-aware training pipeline

  • Lecture 10 - 

    • AI-driven HPC job scheduler

• Module 4 - Large Language Model System

  • Lecture 11 - 

  • Lecture 12 - 

  • Lecture 13 - 

  • Lecture 14 - 

  • Lecture 15 - 

    • LLM assistant for scientific simulation workflows

Late Days Policy

• 5 late days total for the semester

• Maximum 2 late days per assignment

• After late days are exhausted, no submissions accepted

In general, we do not grant extensions on assignments. There are several exceptions:

• Medical emergencies

• Family/personal emergencies

• University-approved travels