RocksDB 8.x benchmarks: large server, cached database

This post has results for performance regressions in all versions of 8.x from 8.0.0 to 8.9.2 using a large server. In a previous post I shared results for RocksDB 8.0 to 8.8 on the same hardware.

tl;dr

• The regressions from 8.0 to 8.9 are small. For the non read-only benchmark steps, QPS from 8.9.2 is at most 3% less than 8.0.0 with leveled and at most 1% less with universal.

I focus on the benchmark steps that aren't read-only because they suffer less from noise.  These benchmark steps are fillseq, revrangewhilewriting, fwdrangewhilewriting, readwhilewriting and overwriteandwait. I also focus on leveled more so than universal, in part because there is more noise with universal but also because the workloads I care about most use leveled.

Builds

I compiled with gcc RocksDB 8.0.0, 8.1.1, 8.2.1, 8.3.3, 8.4.4, 8.5.4, 8.6.7, 8.7.3 and 8.8.1 and 8.9.2 which are the latest patch releases.

Benchmark

All tests used a server with 40 cores, 80 HW threads, 2 sockets, 256GB of RAM and many TB of fast NVMe SSD with Linux 5.1.2, XFS and SW RAID 0 across 6 devices. For the results here, the database is cached by RocksDB. The benchmark was repeated for leveled and universal compaction.

Everything used the LRU block cache and the default value for compaction_readahead_size. Soon I will switch to using the hyper clock cache once RocksDB 9.0 arrives.

I used my fork of the RocksDB benchmark scripts that are wrappers to run db_bench. These run db_bench tests in a special sequence -- load in key order, read-only, do some overwrites, read-write and then write-only. The benchmark was run using 24 threads. How I do benchmarks for RocksDB is explained here and here. The command line to run them is: 

bash x3.sh 24 no 3600 c40r256bc180 40000000 4000000000 byrx

A spreadsheet with all results is here and performance summaries are here for leveled and for universal.

Results: leveled

For the non read-only benchmark steps, QPS from 8.9.2 is at most 3% less than from 8.0.0.

Results: universal

For the non read-only benchmark steps, QPS from 8.9.2 is at most 1% less than from 8.0.0.