Checking jemalloc versions for RSS regressions with MyRocks and the Insert Benchmark

I previously shared a blog post with results for jemalloc, tcmalloc and glibc malloc vs MyRocks and the Insert Benchmark. The summary was to use jemalloc or tcmalloc with MyRocks, or anything that uses RocksDB, because they are much better than glibc malloc at reducing the peak RSS.

Today I share results for the same benchmark using jemalloc versions from 4.0.4 to 5.3.0.

tl;dr

1. There was a big regression WRT peak RSS in jemalloc 4.4.0 and 4.5.0
2. It was mostly fixed in 5.0.1 and 5.1.0
3. It was fixed more in 5.2.0
4. It was fully fixed in 5.3.0

Benchmarks

My previous post explains the benchmark and HW. New details here is that I compiled many versions of jemalloc from source -- 4.0.4, 4.1.1, 4.2.1, 4.3.1, 4.4.0, 4.5.0, 5.0.1, 5.1.0, 5.2.0, 5.3.0 -- and repeated the benchmark for each. This was done twice using different sizes for the RocksDB block cache: 40G and 60G. And then I also repeated things twice -- once using the (older) LRU block cache, once using the (newer) Hyper Clock cache.

I made no attempt to tune or configure jemalloc. Perhaps I will learn whether that was a big deal but I already have more than enough tuning to manage in the DBMS and I prefer to not tune the allocator.

Results

All numbers are here - see lines 16 to 67.

The summary is:

1. There was a big regression WRT peak RSS in jemalloc 4.4.0 and 4.5.0
2. It was mostly fixed in 5.0.1 and 5.1.0
3. It was fixed more in 5.2.0
4. It was fully fixed in 5.3.0