Python 2 has officially completed its life as of Jan 1st, 2020 and the Python 2.x branch will not be maintained anymore. You might be seeing repeated notices of deprecation in log files and terminals like:
1 | DEPRECATION: Python 2.7 will reach the end of its life on January 1st, 2020. Please upgrade your Python as Python 2.7 won't be maintained after that date. |
Python 2 had a wonderful life, coming from a humble background and becoming one of the most popular languages and the most popular scripting language. It’s no wonder Patroni-like tools are developed in Python.
Thanks to the growing popularity of Patroni, there are a large number of Patroni cluster setups across different companies. Now “How to migrate from Python 2 to Python 3 with the least possible downtime” becomes a hanging question for Operations teams and DBAs. Additionally, chances are that the cluster will be running an older version of Patroni. If so, the upgrade plan can take up both these challenges at the same time:
- Upgrade Patroni Software to Latest release
- Switch to Python 3 from Python 2
This blog post is to demonstrate how easy it is, and Patroni’s High Availablity features will be leveraged when we upgrade individual nodes. There are a few key points to note.
- We should not be disturbing the default Python 2 installation (its binaries and modules) because there could be other programs using them.
- Python 2 and Python 3 installations will go to different directories so they can coexist in almost all Linux distros.
- We should avoid multiple Patroni installations (different versions) in a node because they will cause confusion in the long run. So we must remove existing ones.
- Patroni in each node of the cluster can be using different versions of Python because Patroni is not interfering with PostgreSQL replication.
- Since Python precompiles scripts it loads to memory for execution, we can replace the Python script while the corresponding program is running in a loop.
Verification of the Version
We shall verify the version of Python and Patroni we use before proceeding. For example:
1 2 3 | $ ps -eaf | grep patroni postgres 17418 17066 0 10:19 pts/0 00:00:01 /bin/python /usr/bin/patroni /etc/patroni/patroni.yml postgres 17494 17470 0 10:23 pts/1 00:00:00 grep --color=auto patron |
As we see in this node, Patroni invoked Python from /bin/python. Commonly, this will be a symbolic link to the actual Python binary of a specific version.
1 2 | $ ls -l /bin/python lrwxrwxrwx. 1 root root 7 Dec 23 10:39 /bin/python -> python2 |
We won’t be replacing this because it may break other Python-based programs in the system. It’s worth noting the exact Python version.
1 2 | $ /bin/python --version Python 2.7.5 |
Upgrade
Since Python 3 binaries go to a different location in the system, there is nothing stopping us from installing it. It is not going to affect Patroni anyway. I would suggest using the Python from the repository of the Linux distribution.
On CentOS/RHEL, you can use the yum package manager for installing Python binaries and development packages and C compiler as follows:
1 | sudo yum install -y python3-pip pyOpenSSL python3-devel gcc |
The above-mentioned step may change depending on the Linux distribution. For example, on Ubuntu, you may install
1 | sudo apt install python3-pip |
and for the remaining installation, we can use pip for Python 3 (pip3) and execute the following commands one by one:
1 2 3 | sudo pip3 install --upgrade setuptools sudo pip3 install psycopg2-binary sudo pip3 install wheel |
These steps may remain the same for all environments.
IMPORTANT: In this blog post, steps are demonstrated with a fresh, minimal installation of Python 3. Mixing pip installation with the package manager of Linux distribution can be dangerous. Please follow either of the ways to install python packages/modules. It is recommended to avoid installing additional Python packages in a Patroni node as the root user.
Now we are entering the tricky part: uninstalling and installing Patroni while it is running. This is possible because Python does a precompilation of Python code and loads it to memory for execution. So replacing the Python source is almost equivalent to replacing a source file of a compiled program.
The Patroni in the system might have installed in different ways. RPM/DEB packages, Patroni Source code from Git repo of the project, or pip installation. Use the corresponding uninstall method for the same. In case your installation is using RPM/DEB packages, the package managers like YUM or APT provides their own removal options
Note: Percona Distribution for PostgreSQL provides RPM and DEB packages for installation of Patroni.
Even if the existing installation of Patroni is installed using easy_install or source install, pip can help you to remove it.
1 2 3 4 5 6 7 8 9 10 11 | $ sudo pip uninstall patroni DEPRECATION: Python 2.7 will reach the end of its life on January 1st, 2020. Please upgrade your Python as Python 2.7 won't be maintained after that date. A future version of pip will drop support for Python 2.7. More details about Python 2 support in pip, can be found at https://pip.pypa.io/en/latest/development/release-process/#python-2-support Uninstalling patroni-1.5.3: Would remove: /usr/bin/patroni /usr/bin/patroni_aws /usr/bin/patroni_wale_restore /usr/bin/patronictl /usr/lib/python2.7/site-packages/patroni-1.5.3-py2.7.egg Proceed (y/n)? y Successfully uninstalled patroni-1.5.3 |
As we can see in this case, we uninstalled an older version 1.5.3. It’s good practice to make sure that Patroni no longer exists in the previous location:
1 2 | head /usr/bin/patroni head: cannot open '/usr/bin/patroni' for reading: No such file or directory |
Now we can install the new version of Patroni with Python 3. As of writing this blog, the latest release is 1.6.3.
1 2 3 4 5 6 7 | $ sudo pip3 install patroni[etcd] WARNING: Running pip install with root privileges is generally not a good idea. Try `pip3 install --user` instead. Collecting patroni[etcd] Using cached https://files.pythonhosted.org/packages/8f/35/37eeb1b7d54a7d47205050bb6415a74fc7db7a3128e584fc54ff0f5b4cc2/patroni-1.6.3-py3-none-any.whl ... Installing collected packages: patroni Successfully installed patroni-1.6.3 |
The new location of Patroni can be different, but make sure to verify the Python version:
1 2 3 4 | $ which patroni /usr/local/bin/patroni [postgres@pg1 ~]$ head /usr/local/bin/patroni #!/usr/bin/python3 |
During this entire operation of uninstalling and reinstallation of the new version of Patroni, old Patroni will be running. No shutdown is required until this stage. But now the time has come and it’s a matter of new Patroni taking over the PostgreSQL.
Make sure that new Patroni is in the path and service files are updated accordingly.
1 2 | $which patroni /usr/local/bin/patroni |
Now we should be stopping the Patroni service and starting it with new Patroni. Most administrators prefer to manage the Patroni service via Systemd. In case your environment is also managed by systemd, the following steps can be used. As the first step, verify the systemd service configuration (unit file). If Patroni is located in a different directory, please do the required changes to the path.
You can open the systemd service in your favorite text editor:
1 | sudo vi /etc/systemd/system/patroni.service |
However, the best way to edit the service configuration file is to use systemctl’s edit option like
1 | sudo systemctl edit --full patroni.service |
which will open the service configuration for you for editing.
There will be a line mentioning the command to be executed to start the service (ExecStart). Edit this as per the Patroni location.
1 | ExecStart=/usr/local/bin/patroni /etc/patroni/patroni.yml |
Once editing is completed, reload the configuration:
1 | sudo systemctl daemon-reload |
Now we shall restart the Patroni deamon. This will cause the associated PostgreSQL instance to go down and start up again.
1 | sudo systemctl restart patroni |
Finally, we shall verify that the node joined back to the cluster with new Patroni.
1 2 3 4 5 6 7 8 | -bash-4.2$ patronictl -c /etc/patroni/patroni.yml list +----------+--------+------+--------+---------+----+-----------+ | Cluster | Member | Host | Role | State | TL | Lag in MB | +----------+--------+------+--------+---------+----+-----------+ | pgcluser | pg0 | pg0 | Leader | running | 2 | | | pgcluser | pg1 | pg1 | | running | 2 | 0 | | pgcluser | pg2 | pg2 | | running | 2 | 0 | +----------+--------+------+--------+---------+----+-----------+ |
We may explicitly verify that the Patroni is started with Python 3 and we have the latest version:
1 2 3 4 5 6 | $ ps -eaf | grep patroni postgres 16899 1 0 11:24 ? 00:00:02 /usr/bin/python3 /usr/local/bin/patroni /etc/patroni/patroni.yml postgres 16951 16462 0 11:42 pts/0 00:00:00 grep --color=auto patroni $ /usr/local/bin/patroni --version patroni 1.6.3 |
This blog post demonstrates how the upgrade of Python and Patroni can be achieved with the least disruption to service. Only Patroni in one node of the cluster needs to be restarted at a time. We shall repeat the above-mentioned steps in other nodes, also one by one, to upgrade the entire cluster.
Have you tried to put Patroni into a pause mode ?https://patroni.readthedocs.io/en/latest/pause.html
It will allow to shut down patroni daemon on all cluster members without stopping the Postgres instance it is managing.
Hey Jobin, I have some irresistible questions. Is Python a software like wordpress which needs to be updated regularly or is a programming language? This question came to my mind because you said earlier “Python 2 has officially completed its life as of Jan 1st, 2020 and the Python 2.x branch will not be maintained anymore.” What the Python really is that needs to be maintained and who maintains it.
Sorry for the dumb questions.
Thank you Kostiantyn, for that valuable comment. very much appreciate that.
It will be very useful for everyone who is using Patroni. I shall try that option and write about.
Thank you once again.
Jobin.