Thursday, February 16, 2023

CMAKE_BUILD_TYPE: RelWithDebInfo vs Release for MySQL

What is the impact of CMAKE_BUILD_TYPE for MySQL? I have been spending time over the past few weeks attempting to confirm that the way I build MySQL is reasonable.

The MySQL Reference manual lists the choices as: RelWithDebInfo, Debug. It neglects to mention Release. The manual also mentions BUILD_CONFIG which I will try soon.

The cmake online docs don't say much about the impact of Debug, RelWithDebInfo and Release.

So I will diff the compiler command lines for mysqld.cc from MySQL 8.0.32 for builds that use RelWithDebInfo and Release for CMAKE_BUILD_TYPE. All of this was done on a server with Ubuntu 22.04, cmake 3.22.1 and gcc 11.3.0.

tl;dr

  • Where does -fstack-protector-strong come from with RelWithDebInfo? I searched MySQL and cmake source and did not find it.
  • What is the performance impact from -fstack-protector-strong?

Cmake command lines

I tried 3 cmake command lines. The first uses RelWithDebInfo. The second uses Release and adds -g1 to CFLAGS and CXX_FLAGS. The third uses Release as-is without adding -g1.

# Uses RelWithDebInfo
cmake .. \
  -DCMAKE_BUILD_TYPE=RelWithDebInfo \
  -DWITH_SSL=system \
  -DWITH_ZLIB=bundled \
  -DMYSQL_MAINTAINER_MODE=0 \
  -DENABLED_LOCAL_INFILE=1 \
  -DCMAKE_INSTALL_PREFIX=$1 \
  -DWITH_BOOST=$PWD/../boost \
  -DWITH_NUMA=ON \
  -DWITH_ROUTER=OFF \
  -DWITH_MYSQLX=OFF \
  -DWITH_UNIT_TESTS=OFF

# Uses Release and addds -g1
BF=" -g1 "
CF=" $BF "
CXXF=" $BF "
BF=" -g1 "
CF=" $BF "
CXXF=" $BF "
cmake .. \
  -DCMAKE_BUILD_TYPE=Release \
  -DWITH_SSL=system \
  -DWITH_ZLIB=bundled \
  -DMYSQL_MAINTAINER_MODE=0 \
  -DENABLED_LOCAL_INFILE=1 \
  -DCMAKE_INSTALL_PREFIX=$1 \
  -DWITH_BOOST=$PWD/../boost \
  -DCMAKE_CXX_FLAGS="$CXXF" -DCMAKE_C_FLAGS="$CF" \
  -DWITH_NUMA=ON \
  -DWITH_ROUTER=OFF \
  -DWITH_MYSQLX=OFF \
  -DWITH_UNIT_TESTS=OFF

# Uses Release but does not add -g1
cmake .. \
  -DCMAKE_BUILD_TYPE=Release \
  -DWITH_SSL=system \
  -DWITH_ZLIB=bundled \
  -DMYSQL_MAINTAINER_MODE=0 \
  -DENABLED_LOCAL_INFILE=1 \
  -DCMAKE_INSTALL_PREFIX=$1 \
  -DWITH_BOOST=$PWD/../boost \
  -DWITH_NUMA=ON \
  -DWITH_ROUTER=OFF \
  -DWITH_MYSQLX=OFF \
  -DWITH_UNIT_TESTS=OFF

Compiler command line diff: RelWithDebInfo vs Release+g1

The interesting differences are:

  • RelWithDebInfo adds -fstack-protector-strong. I searched source code for both MySQL and cmake and could not find what adds -fstack-protector-strong. That option is explained in gcc docs and LWN has an interesting discussion of it.
  • RelWithDebInfo uses -O2 while Release+g1 uses -O3
  • The size of mysqld is 255M with RelWithDebInfo vs 196M for Release+g1
I did the following to generate these diffs where o.mk has verbose build output:
grep mysqld\.cc o.mk  | grep c++ > cl.mysqld_cc.$d
cat cl.mysqld_cc.$d | tr ' ' '\n' | sort | uniq | \
    grep -v isystem > cl.mysqld_cc.s.$d
diff cl.mysqld_cc.s.build.$a cl.mysqld_cc.$b

"<" means RelWithDebInfo adds it
">" means Release adds it

< -D_FORTIFY_SOURCE=2
< -ffat-lto-objects
< -ffile-prefix-map=...
< -flto=auto
< -fstack-protector-strong
< -g

< -O2
---
> -O3

< -Wdate-time
< -Werror=format-security
< -Wformat

Compiler command line diff: Release and -g1 vs Release

This worked as expected and -g1 was added. The size of mysqld is 196M for Release+g1 vs 76M for Release.

"<" means "Release and -g1" adds it
< -g1

Update 1: Trying -O2 with Release builds

The optimization level is -O3 with CMAKE_BUILD_TYPE=Release. You can use -O2 but don't forget to add -DNDEBUG or the build will fail. See bug 110226 for details.
-DCMAKE_C_FLAGS_RELEASE="-O2 -DNDEBUG" \
-DCMAKE_CXX_FLAGS_RELEASE="-O2 -DNDEBUG" \
Update 2: Wasting time trying -O2 with Release build in 5.7

The advice in the previous section doesn't work with 5.7.40 and -O3 will still be used because there is magic in cmake/build_configurations/compiler_options.cmake to clobber it. That isn't friendly, and the comment in that code isn't correct -- this isn't just done for RelWithDebInfo. Fortunately, that code no longer exists in modern 8.0. My temp workaround was to delete that code and get a Release build that uses -O2.

4 comments:

  1. So, from cmake point of view, the easiest experiment you can do on your own , to check flags it adds for the BULD_TYPE, is to create a dummy hello-world project. Compile it with VERBOSE=1, or cmake --build . --verbose, that will work whether you use ninja or make. Afaik, the cmake's own flags are -O2 (Release), -O2 -g (RelWithDebInfo), -O1 (MinSizeRel) and -g (Debug) . MySQL and MariaDB will add own stuff to the predefined flags, or change them sometimes. They could have also define own build types, but chose not to.

    ReplyDelete
    Replies
    1. I will try BUILD_TYPE soon, just need an idle machine and both are busy with sysbench and the insert benchmark. Later today I will add a third server to my Beelink cluster.

      Delete
    2. Did a build with -DBUILD_CONFIG=mysql_release and the compiler options didn't change. That doesn't surprise me after reading: cmake/build_configurations/mysql_release.cmake

      Delete
  2. The answer from an expert (Yura Surokin of Percona) is:

    '-fstack-protector-strong' comes from this commit https://github.com/mysql/mysql-server/commit/38474c44e5330c0cbd338ee10613d4d3e703e672
    Basically, it introduces a new CMake option 'WITH_PACKAGE_FLAGS' (ON by default for RelWithDebInfo), which, when enabled, automatically adds the output of the 'dpkg-buildflags' / 'rpm --eval %optflags' to CFLAGS / CXXFLAGS.
    On my Ubuntu 22.04 Jammy, for instance,
    CXXFLAGS=-g -O2 -ffile-prefix-map=/home/yura/ws/percona-server=. -flto=auto -ffat-lto-objects -flto=auto -ffat-lto-objects -fstack-protector-strong -Wformat -Werror=format-security

    From:
    https://www.linkedin.com/feed/update/urn:li:activity:7029876032086118400?commentUrn=urn%3Ali%3Acomment%3A%28activity%3A7029876032086118400%2C7032030298649333760%29&replyUrn=urn%3Ali%3Acomment%3A%28activity%3A7029876032086118400%2C7033473230988296192%29&dashCommentUrn=urn%3Ali%3Afsd_comment%3A%287032030298649333760%2Curn%3Ali%3Aactivity%3A7029876032086118400%29&dashReplyUrn=urn%3Ali%3Afsd_comment%3A%287033473230988296192%2Curn%3Ali%3Aactivity%3A7029876032086118400%29

    ReplyDelete

I recently published results with a summary of HW performance counters for sysbench with MySQL on four CPU types. The performance reports we...