1. Introduction to parallel and distributed computing <lect1>
2. Parallel and distributed architectures <lect2>
3. Parallel algorithms <lect3> <ref>
4. Performance modeling <lect4> <video> <youtube> <youtube>
5. Cluster computing with MPI <communication> <example>
6. Big data platform: Hadoop, MapReduce programming, Hadoop Ecosystems <bigdata> <hadoopintro>
Resources:
--hadoop 2 nodes: setup
https://www.linode.com/docs/databases/hadoop/how-to-install-and-set-up-hadoop-cluster/
--pyspark distributed computing (https://www.geeksforgeeks.org/introduction-pyspark-distributed-computing-apache-spark/)
7. Multicore programming with OpenMP
example programs
Write your own code for sobel_edge_filter, threshold, hysteresis .
( See ndimage.convolve.)
see http://www.cpe.ku.ac.th/~cnc/img_test.zip .
see www.cpe.ku.ac.th/~cnc/img_test1.zip
- use three schedules: static (N/#numthread), dynamic(1), guided dynamic
measure time for each schedule
create graph: y axis is time , x axis= numthread = 4,8,16 : (for each algorithm, image file ( yaya_large, kim_jong, yaya_big)) legend = seq / static,1/ dynamic,1 / guided
8. GPU Programming with CUDA
Fine tuning Parallel and GPU programs
colab_cuda1 (hw1,hw2)
Ref: https://sites.google.com/a/ku.th/gpu/home/gpu-tuning-matrixmul-example
https://en.wikipedia.org/wiki/Thread_block_(CUDA_programming)
9. Deep learning acceleration with GPUs ( Caffe,PyTorch, Tensorflow)
and Distributed computing
deep learning distributed <video>
intro to caffe <video>
intro to tensorflow <video> <tensorboard>
intro to pytorch <video> <git hub>
pytorch tensorboard <video>
pytorch data parallel <video> <github> <github>
tensorflow distributed <video>
More: https://sites.google.com/a/ku.th/parallel-computing/deep-learning-gpus
10. Machine learning tools with GPUs/CUDA
python dask <video1> <video2> <github>
RAPIDS AI
(https://github.com/rapidsai/notebooks)
(https://rapids.ai/start.html)
(https://github.com/nalepae/pandarallel)
11. Project & Case study embedded systems with Raspberry Pi/ Jetson