Lately, the number of questions about how to build Percona software has been increased. More and more people try to add their own patches, add some modifications, and build software by themselves. But this raises the question of how to do this in the same way as Percona does, as sometimes the compiler flag can make a drastic impact on the final binary.
First of all, let’s talk about the stages of compiling software.
I would say that at the beginning you need to prepare the build environment, install all the needed dependencies, and so on. For each version, the dependency list would be different. How do you get the correct dependency list? You can get all build requirements from the spec file (on rpm-based systems) or from the control file( on deb-based systems).
The next stage is to get the source code of Percona Server for MySQL. You can do it in different ways:
- Get source tarball from the website
We publish source tarball for each release we issue and you can easily get it for any released version (this is the link for the latest 8.0.22 version).
- Download tarball from GitHub
Here is the link: https://github.com/percona/percona-server/releases/tag/Percona-Server-8.0.22-13
Once you have prepared the build environment, you need to decide what result you should get: binary tarball, rpm, or deb package.
- Binary tarball – you can follow these instructions: https://www.percona.com/doc/percona-server/LATEST/installation.html#compile-from-source and you would be able to build it.
- You also can find, in our manual, how to build deb package: https://www.percona.com/doc/percona-server/LATEST/installation.html#building-percona-server-debian-ubuntu-packages
- And in another blog, you can read about how to build the rpm package: https://www.percona.com/blog/2017/01/20/how-to-manually-build-percona-server-rpm-packages/
So everything looks easy. But what is the way we use, internally, to prepare our release packages? As I mentioned earlier, each compiler option can make a significant effect.
Everyone tries to make life easier and automate all tasks. It is great, as automation is one of the keys to success as you can work on other tasks once the release build is in progress. So we have created a builder script that is used for making builds. It can be found in the Percona Server GitHub repo and can be used by anyone to make his own builds.
This script can install all needed dependencies for the build environment, create binary tarballs, source RPMs and debs, RPMs, debs itself, and the binary tarball. So it covers all build cycles. So how do you use it?
The script has various params:
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 | --builddir=DIR Absolute path to the dir where all actions will be performed --get_sources Source will be downloaded from github --build_src_rpm If it is 1 src rpm will be built --build_source_deb If it is 1 source deb package will be built --build_rpm If it is 1 rpm will be built --build_deb If it is 1 deb will be built --build_tarball If it is 1 tarball will be built --install_deps Install build dependencies(root privileges are required) --branch Branch for build --repo Repo for build --rpm_release RPM version( default = 1) --deb_release DEB version( default = 1) --debug Build debug tarball |
So let’s see how we can make the build process easier:
1. Download build script:
1 | wget https://raw.githubusercontent.com/percona/percona-server/8.0/build-ps/percona-server-8.0_builder.sh |
2. Create a build directory where you are going to perform all build actions:
1 | mkdir /tmp/BUILD_PS |
3. Install dependencies(please note root permissions are required):
1 | sudo ./percona-server-8.0_builder.sh --builddir=/tmp/BUILD_PS --install_deps=1 |
4. Download source code:
– From Percona repo (it is used by default):
1 | ./percona-server-8.0_builder.sh --builddir=/tmp/BUILD_PS --get_sources=1 --branch=Percona-Server-8.0.22-13 |
– From your own repo and branch:
1 | ./percona-server-8.0_builder.sh --builddir=/tmp/BUILD_PS --get_sources=1 --branch=<your_branch_name> --repo=<link_to_your_repo_on_github> |
5. Prepare src rpm:
1 | ./percona-server-8.0_builder.sh --builddir=/tmp/BUILD_PS --build_src_rpm=1 |
Please note if you already have your source tarball, just create a directory named source_tarball in the build directory and put it into it.
6. Prepare source deb:
1 | ./percona-server-8.0_builder.sh --builddir=/tmp/BUILD_PS --build_source_deb=1 |
Please note if you already have your source tarball, just create a directory named source_tarball in the build directory and put it into it.
7. Prepare rpm:
1 | ./percona-server-8.0_builder.sh --builddir=/tmp/BUILD_PS --build_rpm=1 |
8. Prepare deb:
1 | ./percona-server-8.0_builder.sh --builddir=/tmp/BUILD_PS --build_deb=1 |
9. Prepare tarball:
1 | ./percona-server-8.0_builder.sh --builddir=/tmp/BUILD_PS --build_tarball=1 |
So as you can see, the build procedure becomes easier and you don’t need to think about what dependencies are needed, what cmake params should be used, etc. This build script will make everything for you and will use all build flags (if you didn’t change them in your sources) that we use for release builds.
This a great help, thank you!. Could we make some minor tweaks?
The repo is quite big, could we clone only the branch we are going to use as base? (git clone -b branch –single-branch)
Could we add RHEL8 support to the script? It looks like is not contemplating it (not disabling mysql module and pulling jemalloc v5).
I would also like to see future blog posts about the following steps: how to format code prior contributions, what to do when your contribution is ready, what to expect during reviews…
Hi Francisco.
Thanks for your feedback.
Sure you can clone only branch. And script works well for RHEL8. We prepare packages for RHEL8 using this script and the same approach.
Wouldn’t it be easier to just share the Dockerfiles that are perhaps used in your CI to build each new version? Like it’s e.g. done in Manticore Search – https://github.com/manticoresoftware/manticoresearch/tree/master/dist/build_dockers
With the script there’s still a higher risk of an issue since you don’t control what is in the system before you run the script. With an image/container you know for sure the environment will be 100% the way it’s supposed to be.