Tuesday, October 31, 2023

Postgres vs MySQL: the impact of CPU overhead on performance

This post documents how MySQL and Postgres performance has changed over time using sysbench. It compares QPS for Postgres 11.21 vs MySQL 5.6.21 and then Postgres 16.0 vs MySQL 8.0.34.

The focus here is on CPU overhead and for that a workload that is low-concurrency, CPU-bound and cached is sufficient. However a full comparison would also include results for workloads with more concurrency and workloads that are IO-bound.

The benchmark was explained in a series of posts - see part 1part 2part 3part 4 and part 5 and the results here are from my small server. Usually I avoid direct comparisons between Postgres and MySQL but I am doing this because I care about the future of MySQL and the CPU regressions in it over time have become a problem.

tl;dr

  • Postgres 11.21 and MySQL 5.6.21 have similar QPS on most sysbench tests
  • Postgres 16.0 gets much more QPS than MySQL 8.0.34 on most sysbench tests
  • The problem for MySQL is new CPU overhead

Results

The spreadsheet with all of the numbers is here.

The charts below have two bars per sysbench test. Both bars show relative QPS.
  • The first (blue) bar has (QPS for Postgres 11.21 / QPS for MySQL 5.6.21)
  • The second (red) bar has (QPS for Postgres 16.0 / QPS for MySQL 8.0.34)
Note that the red bars tend to be taller than the blue bars because per-query CPU overheads have gotten larger from MySQL 5.6 to 8.0 while they have not from Postgres 11 to 16.

3 comments:

  1. Whether the starting point is 5.6.21 released in 2014 or 5.6.51 released in 2021, the result is still the same -- MySQL suffers from large perf regressions courtesy of new CPU overhead in 5.7.x and 8.0.x

    On the bright side, Postgres from version 11 to 16 (~2018 to today) does not suffer from such CPU perf regressions.

    The release cycles might never match up because MySQL major releases have been lingering for much longer than Postgres.

    ReplyDelete
  2. Will mysql 4x speed improvement on 8.0.35 innodb insert-heavy tests affect this ?

    ReplyDelete
    Replies
    1. It might help a few of the microbenchmarks, but I have only been running cached workloads for sysbench because I use it to search for CPU issues. The fix helps IO-bound workloads more than CPU-bound: don't want the optimizer to do too many disk reads.

      But 102037 was fixed in 8.0.31 and that helps sysbench
      https://smalldatum.blogspot.com/2023/10/perf-regressions-in-mysql-from-5621-to_30.html

      Delete