HPC

Modern computing architectures. (Top-left) Modern computers ship with several multicore chips (typically 2–4) configured to share a centralized memory. Each multicore chip contains several computing cores (2–6) sharing a cache memory (typically the third level, L3). Internally, each core contains two more cache levels (L1 and L2, not shown in this figure). (top-right) Cluster of multicore computers. Each node has m processors sharing a single centralized memory. The nodes are then connected through an interconnection network. Most current supercomputers are also based on this architectural model. In this case, the so-called distributed-shared memory (DSM) architecture may be available, whereby there is a virtually unique memory system but there is a non-uniform memory access (NUMA), depending on the physical location of the data. (Bottom-left) Graphics Processing Units (GPUs) are composed by several Streaming Multiprocessors (SM) (e.g. 30 and 16 in the second and third generation of NVIDIA GPUs, respectively). Each SM is made up of a number of cores (8 and 32, respectively) that share a register file and a local memory. All the SMs share the global device memory. In the third generation, a hierarchy of cache memory is provided. In particular, a L2 cache level is between the SMs and the device memory. (bottom-right) Hybrid CPU + GPU computing on a computer equipped with multicore processors and multiple GPUs. The system keeps a pool of tasks to do. A number of threads to be mapped to CPU cores (denoted by C-threads) are running concurrently in the system. Also, specific threads (denoted by G-threads) in charge of the tasks to be computed on the GPUs are also running. The tasks are asynchronously dispatched to the threads on-demand. In the figure, allocation of tasks to threads are color-coded. Note that the G-threads will request tasks more often than the C-threads as GPUs make the calculations faster than a single CPU core. Moreover, faster GPUs will be assigned work more frequently than modest GPUs.