Friday, September 20, 2024

Ubuntu 22.04, Linux kernels and me

This is a summary of interesting performance issues I have had with the HWE kernels (6.5.x) on Ubuntu 22.04, mostly this year. This is mostly a note to my future self.

  • Use ext4 instead of XFS
    • I have used XFS since pre-2010 because it avoided a few perf problems that show up in the ext family. But I finally found a reason to switch to ext4. The problem is that with 6.5 kernels (when HWE is used) there are perf problems and many error messages (WQ_UNBOUND, CPU hogging). Some details are here. I am curious if XFS needs a few updates to match whatever changed in the 6.5 kernels.
  • Switch from schedutil to performance CPU frequency governor
    • This issue hurt MyRocks a lot more than Postgres or InnoDB. I don't know the root cause but MyRocks perf was lousy with the schedutil CPU frequency governor. See here for more details from me, and this LWN post might be relevant.
  • readahead requests are dropped
    • RocksDB changed how readahead is done when O_DIRECT is not used and the change uses the readahead call. It looks like readahead requests are ignored when the request size is larger than the value of max_sectors_kb. Alas, to figure out the value of max_sectors_kb you must read /sys/block/$device/queue/max_sectors_kb and I prefer to avoid that (whether I have to do it, or RocksDB code does it). Surely there is a better way to handle this.


No comments:

Post a Comment

RocksDB on a big server: LRU vs hyperclock, v2

This post show that RocksDB has gotten much faster over time for the read-heavy benchmarks that I use. I recently shared results from a lar...