GPUs nodes

What is GPU computing?

Graphics processing unit (GPU) were originally built to to handle computer graphics processing. There is increasing use of these to perform computational tasks previously handled by the central processing unit (CPU). GPUs operate at lower frequencies than CPUs, but have many more cores. Certain computational tasks can achieve magnitudes of performance improvement with GPU-CPU framework.

Coeus himem GPU nodes

The 2 available himem on the Coeus (high memory nodes) were out-fitted with a Nvidia Tesla V100 GPU.

Himem nodes

  • 2 x Intel(R) Xeon(R) CPU E5-2650 v4 @ 2.20GHz. 24 cores (2 x 12)

  • 768 GB ECC RAM

  • 1.9TB local scratch disk

  • Tesla V100 GPU

  • OPA 100Gbps networking

Tesla V100 specifications

  • 5,120 CUDA cores

  • 640 Tensor Cores

  • 7 TeraFLOPS double-precision performance

  • 14 TFLOPS single-precision performance

  • 112 Tensor performance

  • 900 GB/s memory bandwidth with CoWoS HBM2 Stacked Memory

  • 32 GB of HBM2 ECC memory

  • Compute APIs CUDA, DirectCompute, OpenCL™, OpenACC


How to use GPU nodes in the Coeus cluster

Interactive session

You can start an interactive session as follows:

salloc -N 1 -p himem --gres=gpu:1 srun --pty bash

Batch processing with sbatch example

Here's an sbatch example with a trivial cuda program (https://devblogs.nvidia.com/easy-introduction-cuda-c-and-c/)

#!/bin/bash


#SBATCH --job-name saxpy


#SBATCH --nodes 1

#SBATCH --partition himem

#SBATCH --output saxpy_%j.txt

#SBATCH --error saxpy_%j.err

#SBATCH --gres=gpu:1


srun /home/username/examples/cuda/saxpy