Node Interleaving Benchmarks

NUMA: Non Unified Memory Access

Expert Mode

UMA: Unified Memory Access

Bulletproof Mode

Node Interleaving disabled (NUMA) means:

• Memory is allocated, if undefined, to the first RAM bank available (good luck having it to default to the 2nd CPU)
• If the requested memory is at the right RAM bank the CPU needs, it has the lowest latency possible
• If the requested memory is not at the right RAM bank the CPU needs, it incurs a latency cost (to go through the UPI)

On Skylake-SP, the metrics are approximately the following:

• On the right RAM bank: 85 nanoseconds, 100 GBps
• On the bad RAM bank: 135 nanoseconds, 34.5 GBps

Having NUMA means you know very well how to allocate the memory (numactl in Linux), and how to handle CPU affinity.

As the number of NUMA nodes (number of sockets) increases, the performance without memory allocation optimization decreases significantly.

Node Interleaving enabled (UMA) means:

• Memory is allocated in a round-robin fashion: it just goes... everywhere in a round-robin fashion
• Having the memory at the wrong place does not matter, in theory half of it is at the right place already, half of it is at the wrong place
• It "averages" the latency and the bandwidth penalties of NUMA

On Skylake-SP, the metrics are approximately the following:

• On any bank: 118 nanoseconds, 125 GBps

Having UMA means you do not have to deal with with memory allocation, as memory going anywhere is fine.

As the number of NUMA nodes (number of sockets) theoretically increases, the performance remains invariant.

Example: what happens to the performance if you allocate manually the memory to the right or wrong place?

Locale vs remote NUMA binding: floating point workloads

From:

NUMA Best Practices for Dell PowerEdge 12th Generation Servers

xgboost

• Versions used: commit 8f6aadd
• Flags used:
  • gcc: -O3 -mtune=native

LightGBM

• Versions used: commit 3f54429
• Flags used:
  • gcc: -O3 -mtune=native

xgboost

Installing xgboost directly from R:

devtools::install_github("Laurae2/xgbdl")

xgbdl::xgb.dl(compiler = "gcc", commit = "8f6aadd", use_avx = TRUE, use_gpu = FALSE)

LightGBM

Installing LightGBM directly from R:

devtools::install_github("Laurae2/lgbdl")

lgbdl::lgb.dl(commit = "3f54429", compiler = "gcc")

xgboost

Hyperparameters, average of 5 runs (approximately 48h):

• Depth: 8
• Leaves: 255
• Hessian: 1
• Minimum Loss to split: 0
• Column Sampling: 100%
• Row Sampling: 100%
• Iterations: 10
• Learning Rate: 0.25
• Boosting method: gbdt, fast histogram
• Bins: 255
• Loss function: binary:logistic

Note: the timing takes into account the binning construction time, which is approximately 50% to 70% of the xgboost timing.

It takes 13 minutes with 1 thread, 2 minutes with 64 threads.

LightGBM

Hyperparameters, average of 5 runs (approximately 14h):

• Depth: 8
• Leaves: 255
• Hessian: 1
• Minimum Loss to split: 0
• Column Sampling: 100%
• Row Sampling: 100%
• Iterations: 10
• Learning Rate: 0.25
• Boosting method: gbdt
• Bins: 255
• Loss function: binary

Note: the timing does not take into account the binning construction time.

It takes 16 minutes using 1 thread, 23 seconds using 64 threads.

Use the Performance Analysis if you expect to compare timings data.

Check interactively on Tableau Public:

https://public.tableau.com/views/NodeInterleavingv1/Dashboard?:bootstrapWhenNotified=true&:display_count=y&:display_static_image=y&:embed=y&:showVizHome=no&publish=yes

Provided dynamic and interactive filters:

• Threads