High Performance Computing for Science and Engineering

As manufacturing processes approach the physical limits of transistor density on modern computing architectures, efficient code must leverage parallel execution to scale with available hardware resources. It is therefore crucial for software developers to adopt a “think parallel” mindset, given that computers are a fundamental tool for solving complex scientific problems across academia, industry, and society. This course introduces parallel programming and its relationship to computer architectures. Various forms of parallelism are considered and exploited through several programming models, with a particular focus on shared and distributed memory programming. The techniques are explored in depth with homework, lab sessions, and a term project.

Prerequisites

The course assumes the student is comfortable reading and writing code in C++ and Python. Familiarity with Linux command line tools, particularly ssh and Git, is also expected. While advanced C++ knowledge is not required, CS2050 will not teach the basics of programming, and the class is likely unsuitable for first-year students. A determined student who lacks one or two of these prerequisites might still manage to succeed in the course, but only if they are willing to work hard and use external resources to fill gaps in their knowledge.

Course Objectives

• For students to develop a theoretical and practical understanding of parallel programming models, including those appropriate for shared memory computing (e.g., OpenMP), distributed computing (e.g. MPI), and accelerated computing (e.g. CUDA).

• For students to deepen their programming skills, with special emphasis on tools relevant for cutting edge scientific computing. The exercises and assignments will involve regular use of a compute cluster dedicated to the course. Students will make frequent use of git for managing source code, slurm for submitting jobs to the cluster, CMake for building software,  and further tools for profiling and optimizing code.

Course Format

The course elements are listed below. Percentages in brackets indicate the contribution to the final grade. The class does not have exams.

1. Lectures: Deliver the main content. Some lectures will include interactive exercises. Attendance is required.

2. Quizzes [25%]: Short in-class quizzes each Thursday. Students who remain engaged with the lectures, complete any assigned readings, and finish the homework assignments in a timely manner should not find the quizzes overly difficult.

3. Homework [40%]: Homework assignments, due roughly every two weeks, expand on the lecture material and involve coding exercises. Some problems are easy, with step-by-step instructions, while others require more initiative from the student.

4. Project [35%]: The course project, due at the end of the semester, allows the student (in a small team) to implement methods learned in class in the context of a real application. Topics are proposed by the teams and may involve research problems of individual team members.

The teaching staff will hold regular office hours, where students can review and discuss course materials, as well as more structured sections, where students can receive guidance for some aspects of the homework.

Lecture recordings

Lectures will be recorded and uploaded automatically to Panopto after a few hours.

Quizzes

There will be 12 short quizzes held in class, one during each Thursday lecture, which will consist of multiple-choice and/or short-answer questions. The quizzes are intended to assess individual engagement with the course (including the current lecture, as well as past lectures, readings, and homework assignments).

During quizzes, students may neither collaborate nor use any reference materials (paper or electronic).

The lowest four quiz scores will be dropped when calculating final grades. Furthermore, the quizzes are designed to take no more than five minutes, but ten minutes will be given. Together, these policies are intended to accommodate occasional absences, whether due to illness or other reasons, as well as students receiving extra time (1.5x or 2.0x). Further accommodations will be rare but considered on a case-by-case basis. Please contact the instructors with any questions or concerns.

Homework

Detailed instructions for each homework assignment will be linked from the Deadlines page. 

The assignments give students practical experience writing HPC code and working in cluster environment. Note that the compute cluster is a shared, finite resource, meaning submitted job(s) might not execute immediately and could be delayed by several hours or more. For this reason, students are strongly encouraged to complete assignments as early as feasible, and extensions will not be granted simply because the cluster was busy leading up to a deadline.

Collaboration between students is welcome, and the use of external resources (including AI tools) is encouraged. However, each student must submit their own work and is expected to understand every detail of their submissions.

Students have a budget of four late days, used to submit homework assignments late without penalty (up to four days in total throughout the semester). No more than two late days can be used on a single homework. There are no fractional late days, and they may not be used for project-related submissions. Once a student has exhausted their late days, late homework submissions will be penalized by 25% per day of the total possible points. Except for medical (if accompanied by a doctor’s note) or other official, University-excused reasons.

If a student believes their grade for a homework question is incorrect, after comparing it with the solution, they may submit a regrade request for that question via Gradescope once.

Academic Integrity

The quiz, homework, and project sections give specific expectations for those elements of the course. Students are further reminded of the Honor Code.

Accessibility Statement

Harvard University values inclusive excellence and providing equal educational opportunities for all students. Our goal is to remove barriers for disabled students related to inaccessible elements of instruction or design in this course. If reasonable accommodations are necessary to provide access, please contact the Disability Access Office (DAO). Accommodations do not alter fundamental requirements of the course and are not retroactive. Students should request accommodations as early as possible, since they may take time to implement. Students should notify DAO at any time during the semester if adjustments to their communicated accommodation plan are needed.

Students may also contact the instructors with accessibility questions or concerns.