Parallel Computing
General objective
The primary objective of this course is to introduce students to the essential principles and practices of parallel computing, emphasizing its role as a fundamental tool for accelerating computations and addressing problems demanding huge amounts of memory. Students will understand how parallel computing architectures and techniques can effectively address complex computational problems. Through theoretical and practical approaches, this course aims to equip students with the skills and knowledge necessary to design, analyze, and implement parallel algorithms, preparing them for the evolving demands of computing in various fields.
Course information
Number of sessions: 32
Postgraduate program: Computer Science
Room: 8301 - Tuesday 12:00 - 13:30 and Thursday 12:00 - 13:30
Period: Spring 2024 (January 2024 - May 2024)
Instructors:
Julio César Pérez Sansalvador
Gustavo Rodríguez Gómez
Course content:
Introduction to parallel computing
Moores' law
General concepts - communication, synchronization, granularity, parallel overhead, speed up
Scope and limitations of parallel computing
Parallel systems classifications
Parallel algorithm's design
Automatic parallelization
Partitioning, decomposition, and distribution (data decomposition and task decomposition)
Communication
Agglomeration
Mapping
Load balancing
Communication operators
Point-to-point communication (One-to-one)
Collective communication (All-to-all, All-reduce, Scatter and Gather)
Synchronization (Barriers, Locks, Semaphores)
Parallel computing paradigms
Shared memory
Distributed memory
Advanced topics
Acceleration by hardware
Automatic load balancing
Parallel input and output
Restarting parallel applications
Restarting from crashes or failures
Bibliography
Ananth Grama, Anshul Gupta, George Karypis and Vipin Kumar, Introduction to Parallel Computing, Second Edition, Addison Wesley, 2003.
Peter Pacheco. 2011. An Introduction to Parallel Programming (1st. ed.). Morgan Kaufmann Publishers Inc., San Francisco, CA, USA.
Trobec, R., Slivnik, B., Bulić, P., Robič, B., Introduction to Parallel Computing: From Algorithms to Programming on State-of-the-art Platforms, Series Undergraduate topics in computer science, Springer, 2018.
G. Hager and G. Wellein, Introduction to High-Performance Computing for Scientists and Engineers, Chapman & Hall/CRC Computational Science, CRC Press, 2010.
J. Palach, Parallel Programming with Python, Community experience distilled, Packt Publishing, 2014.
G. Zaccone, Python Parallel Programming Cookbook, Packt Publishing, 2015.