Web Compression

TurboBench: Static/Dynamic web content compression benchmark

file html8 : 100MB random html pages from a 2GB Alexa Top sites corpus.

Number of pages = 1178

Average length = 84886 bytes

The pages (length + content) are concatenated into a single html8 file,

but compressed/decompressed separately using the multiblock mode in TurboBench.

compress: page1,page2,...pageN

decompress : Page1,page2,...pageN

This avoids the cache effects like in other benchmarks, where small files

are processed repeatedly in the L1/L2 cache, showing unrealistic results.

size: 100,000,000 bytes, Content-Encoding: gzip,br,deflate

Single core in memory benchmarks: 64 bits gcc 8.3 Skylake i6700 3.4 GHz

Size Ratio% C.Time D.Time Compressor C.Mem Peak D.Mem Peak(bytes) MB/s MB/s Peak memory usage bold=Pareto frontier on ratio %

16457826 16.5 0.75 406.67 brotli 11 10,616,800 244,352 br
17985130 18.0 brotli,11V static dictionary disabled
18579017 18.6 53.42 486.18 brotli 5 10,703,288 196,080 br
19615056 19.6 0.17 523.62 zopfli 33,644,120 14,352 gzip
19815500 19.8 6.54 880.07 libdeflate 12 17,938,928 15,184 gzip
19971692 20.0 86.04 495.59 brotli 4 10,145,440 196,416 br
20363869 20.4 34.36 426.98 zlib 9 274,096 14,320 gzip
20363872 20.4 42.36 569.75 zlib-ng 9 274,240 14,320 gzip SSE4.2
20450976 20.5 55.26 397.25 zlib 9 cloudflare
20460336 20.5 95.53 883.85 libdeflate 7 2,202,032 15,184 gzip
20485533 20.5 54.69 424.71 zlib 6 274,096 14,320 gzip
20566179 20.6 114.54 879.36 libdeflate 6 2,202,032 15,184 gzip
20566547 20.6 89.90 395.24 zlib 6 cloudflare
20617904 20.6 78.78 570.72 zlib-ng 6 274,096 14,320 gzip SSE4.2
21627476 21.6 162.00 493.13 brotli 2 9,220,344 228,120 br
22580711 22.6 220.12 918.54 igzip 3 696,336 ? gzip SSE4.2
22630144 22.6 171.76 835.28 libdeflate 1 2,202,032 15,184 gzip
23098360 23.1 540.13 906.77 igzip 2 663,568 ? gzip
23153034 23.1 252.88 482.37 brotli 1 1,193,488 162,528 br
23220008 23.2 172.40 366.74 zlib 1 cloudflare
23287503 23.3 559.30 898.80 igzip 1 565,264 ? gzip SSE4.2
23723266 23.7 123.10 392.15 zlib 1 274,096 14,320 gzip
24070531 24.1 394.82 455.97 brotli 0 146,112 162,528 br
25355327 25.4 624.93 847.08 igzip 0 ? ? gzip SSE4.2
28214601 28.2 399.44 435.54 slz 6 (64k stack+132k) 14,320 gzip
28214601 28.2 399.24 435.54 slz 9 " 14,320 gzip
28463156 28.4 689.51 1912.51 lzturbo 20
29466128 29.5 485.42 491.45 slz 1d " 14,320 deflate
29476316 29.5 403.66 428.32 slz 1 " 14,320 gzip
29994525 30.0 313.26 554.18 zlib-ng 1 274,096 14,320 gzip SSE4.2

C Size ratio% C MB/s D MB/s ODROID-N2-1.8GHz ARM A73-64bits gcc7.4 2019-06-28

16459983 16.5 0.25 154.09 brotli 11
18581144 18.6 15.51 169.32 brotli 5
19615056 19.6 0.05 175.82 zopfli
19815500 19.8 1.92 286.83 libdeflate 12
19974430 20.0 16.01 172.33 brotli 4
20363869 20.4 12.52 187.11 zlib 9
20364099 20.4 12.05 261.80 zlib_ng 9
20460336 20.5 27.48 287.00 libdeflate 7
20485533 20.5 19.63 185.65 zlib 6
20566179 20.6 32.05 286.47 libdeflate 6
20947326 20.9 22.41 254.83 zlib_ng 6
21627476 21.6 34.71 168.34 brotli 2
22630144 22.6 44.53 267.78 libdeflate 1
22635728 22.6 20.18 170.16 igzip 3
23153034 23.2 96.86 161.52 brotli 1
23575500 23.6 63.25 165.87 igzip 2
23646059 23.6 48.23 236.14 zlib_ng 1
23723266 23.7 48.03 172.86 zlib 1
23724297 23.7 68.82 164.33 igzip 1
25769582 25.8 86.27 155.66 igzip 0
28214601 28.2 105.70 187.76 slz 9
28214601 28.2 105.65 187.73 slz 6
28463156 28.5 192.31 596.49 lzturbo 20
29476316 29.5 94.71 185.69 slz 1

Size Ratio% C.MB/s D.MB/s RASPBERRY PI3 - 1.2GHz ARM 64 bits Linux - gcc 6.3 2019-06

16461610 16.5 0.11 67.66 brotli 11
18579017 18.6 6.51 75.93 brotli 5
19815500 19.8 0.84 154.53 libdeflate 12
19971692 20.0 10.41 77.66 brotli 4
20363869 20.4 6.47 99.66 zlib 9
20485533 20.5 10.48 99.16 zlib 6
20566179 20.6 15.04 150.22 libdeflate 6
22630144 22.6 20.36 137.34 libdeflate 1
23143931 23.1 27.99 70.57 brotli 1
23723266 23.7 27.67 91.42 zlib 1
28214601 28.2 46.39 99.36 slz 9
28214601 28.2 46.39 96.36 slz 6
28423085 28.4 199.44 459.34 lzturbo 20 (not gz,br compatible)
29476316 29.5 43.69 98.14 slz 1

Speedup relative to Transfer without Compression (= 100%)

The compression libraries (JUN 2019): brotli igzip: Intel(R) Intelligent Storage Acceleration Library intel SSE4.2+x86-64 asm libdeflate libslz 1.1.0 (2019.03.16) zlib-ng intel SSE4.2 (library libz-ng build w/ make in zlib-ng) zlib cloudflare intel SSE4.2 zlib 1.2.11 (compiled w/ turbobench) zopfli(decompreession w/ zlib-ng)

Plots autmatically generated with turbobench

Last update: 28 JUN 2019