Wednesday, February 15, 2023

Adventures in compiling MySQL

I like to track performance regressions from old to new releases for Postgres and MySQL and as part of this I compile them from source. This is easy and fast for Postgres. Alas, MySQL builds are slower and for older versions the builds doesn't always work without some effort. I didn't try to quantify why modern MySQL is so much slower to compile but the sql/gis directory is a part of the problem and Postgres uses C while MySQL has all of the overheads from C++.

Today I am building MySQL versions 8.0.32, 8.0.31, 8.0.28, 8.0.22, 5.7.40 and 5.6.51 on Ubuntu 22.04. The builds for 8.0.32 and 8.0.31 are easy. For other releases I had to try harder, and a summary of the changes is:

  • 8.0.28
    • Link with OpenSSL version 1 -- I had to install openssl-1.1.1t from source because Ubuntu 22.04 uses OpenSSL 3 and that breaks the MySQL build.
  • 8.0.22
    • Link with OpenSSL version 1
    • Hide components/mysqlbackup because it has compile errors. I hid the directory because I didn't see Cmake command line option to avoid it.
    • Link with OpenSSL version 1
    • Edit sql-common/sql_string.cc to include <limits>
  • 5.7.40
    • Link with OpenSSL version 1
  • 5.6.51
    • Link with OpenSSL version 1
    • Add -DCMAKE_CXX_FLAGS=-""--std=c++11" to avoid compile errors

Notes

The command line for installing OpenSSL:

./config --prefix=$X --openssldir=$X shared ; make ; make install

The 8.0.22 compile errors fixed by hiding components/mysqlbackup

.../mysql-8.0.22/include/mysql/components/services/page_track_service.h:58:36: error: ‘size_t’ has not been declared
../mysql-8.0.22/include/mysql/components/services/page_track_service.h:139:59: error: ‘size_t’ has not been declared
.../mysql-8.0.22/components/mysqlbackup/backup_page_tracker.cc:401:7: error: invalid conversion from ‘int (*)(THD*, const uchar*, size_t, int, void*)’ {aka ‘int (*)(THD*, const unsigned char*, long unsigned int, int, void*)’} to ‘Page_Track_Callback’ {aka ‘int (*)(THD*, const unsigned char*, int, int, void*)’} [-fpermissive]

More compile errors for 8.0.22 from sql-common/sql_string.cc

.../b/mysql-8.0.22/sql-common/sql_string.cc:111:20: error: ‘numeric_limits’ is not a member of ‘std’
  111 |     if (len > std::numeric_limits<uint32>::max()) return true;

... fixed by this diff

--- mysql-8.0.22.orig/sql-common/sql_string.cc  2020-09-23 12:37:48.000000000 +0000
+++ mysql-8.0.22/sql-common/sql_string.cc       2023-02-15 20:32:57.093692768 +0000
@@ -23,6 +23,7 @@
 #include "sql_string.h"

 #include <algorithm>
+#include <limits>

 #include "my_dbug.h"
 #include "my_macros.h"

Cmake command lines

For 8.0.28

cmake .. \
  -DCMAKE_BUILD_TYPE=Release \
  -DWITH_SSL=/home/mdcallag/d/ssl \
  -DWITH_ZLIB=bundled \
  -DMYSQL_MAINTAINER_MODE=0 \
  -DENABLED_LOCAL_INFILE=1 \
  -DCMAKE_INSTALL_PREFIX=$1 \
  -DWITH_BOOST=$PWD/../boost \
  -DWITH_NUMA=ON \
  -DWITH_ROUTER=OFF -DWITH_MYSQLX=OFF -DWITH_UNIT_TESTS=OFF

For 8.0.22

cmake .. \
  -DCMAKE_BUILD_TYPE=Release \
  -DWITH_SSL=/home/mdcallag/d/ssl \
  -DWITH_ZLIB=bundled \
  -DMYSQL_MAINTAINER_MODE=0 \
  -DENABLED_LOCAL_INFILE=1 \
  -DCMAKE_INSTALL_PREFIX=$1 \
  -DWITH_BOOST=$PWD/../boost \
  -DWITH_NUMA=ON \
  -DWITH_ROUTER=OFF -DWITH_MYSQLX=OFF -DWITH_UNIT_TESTS=OFF

For 5.7.40

prefix=$1
cmake .. \
      -DCMAKE_BUILD_TYPE=Release \
      -DCMAKE_INSTALL_PREFIX:PATH=$prefix \
      -DWITH_SSL=/home/mdcallag/d/ssl \
      -DWITH_ZLIB="bundled" \
      -DMYSQL_DATADIR="${prefix}/data" \
      -DMYSQL_UNIX_ADDR="${prefix}/var/mysql.sock" \
      -DENABLED_LOCAL_INFILE=1 \
      -DMYSQL_MAINTAINER_MODE=0 \
      -DWITH_BOOST=$PWD/../boost \
      -DWITH_NUMA=ON \
      -DWITH_UNIT_TESTS=OFF


For 5.6.51

prefix=$1
cmake .. \

      -DCMAKE_BUILD_TYPE=Release \
      -DCMAKE_INSTALL_PREFIX:PATH=$prefix \
      -DWITH_SSL=/home/mdcallag/d/ssl \
      -DWITH_ZLIB="bundled" \
      -DMYSQL_DATADIR="${prefix}/data" \
      -DMYSQL_UNIX_ADDR="${prefix}/var/mysql.sock" \
      -DENABLED_LOCAL_INFILE=1 \
      -DMYSQL_MAINTAINER_MODE=0 \
      -DWITH_PERFSCHEMA_STORAGE_ENGINE=0 \
      -DCMAKE_CXX_FLAGS="--std=c++11" \
      -DCMAKE_UNIT_TESTS=OFF


No comments:

Post a Comment

RocksDB on a big server: LRU vs hyperclock, v2

This post show that RocksDB has gotten much faster over time for the read-heavy benchmarks that I use. I recently shared results from a lar...