Distributed computing and learning

This course mainly focus on distributed-memory parallel framework, where each processor (core, CPU, GPU, IoT device) has its own memory space which can not be shared with others. This parallel framework is popular as it corresponds to many real application scenarios such as the cluster, the sensor network and IoT.

The goal of this course is to explore the most popular strategies for parallelizing machine learning (ML) tasks without data-sharing across various real-world application scenarios. Additionally, the students will learn to maintain the privacy and security of such distributed systems. For the lab, the students will parallelize their ML tasks in a real GPU-equipped cluster (NEF) using PyTorch and Flower. This hand-on experience will enable students to accelerate heavy computations involved in training large neural networks.

Content:

Distributed Learning Algorithms
1. Server-Client Framework
2. Decentralized Framework
Privacy
1. Vulnerabilities in Distributed Systems
2. Differential Privacy Mechanisms
Security
1. Robust Algorithms
2. Security Guarantees

Time schedule for distributed computing course

Google Sites

Report abuse