I want to present a tool for running a specific version of open source databases in a single instance, replication setups, and Kubernetes. AnyDbVer deploys MySQL/MariaDB/MongoDB/PostgreSQL for testing and experiments.
It Could Be Started By…
Docker (or Podman) or dbdeployer (MySQL-Sandbox successor) could also start a specific database version, but such installations are significantly different from production setups.
Ansible Playbook
There is an Ansible playbook with few roles configurable by environment variables.
Bash Scripts
You may ignore the Ansible layer because the ./anydbver bash script hides it.
In LXD containers
It’s not practical to reserve physical servers or virtual machines for all tasks. Frequently you need just a few database instances and the ability to restart it with systemd and check logs with journalctl. AnydbVer spawns one or more Linux Containers managed by Canonical (Ubuntu) LXD, and LXD containers support systemd not using a layered filesystem setup.
Best Performance with Linux Running on Hardware Directly
Linux Containers is not an emulation layer. From the “host” system, you can see all processes and files created by containers. Thus the performance is very similar to the same database running on a physical server.
You Can Run Vagrant + VirtualBox as Well, For Other OS
LXD setup is relatively easy, but sometimes you may need to run AnydbVer just with a few commands. Vagrant could start an unmodified Ubuntu virtual machine and automatically configure AnydbVer for you.
You can find the required files and setup instructions at https://github.com/ihanick/anydbver.
Clone the https://github.com/ihanick/anydbver repository to setup LXD or start Vagrant.
1 2 3 4 5 | git clone https://github.com/ihanick/anydbver.git cd anydbver vagrant up vagrant ssh cd anydbver |
Single Instance Usage
Imagine that you need the exact CentOS 7 package version: Percona Server for MySQL 5.6.43-rel84.3:
1 2 3 4 | $ ./anydbver deploy percona-server:5.6.43-rel84.3 $ ./anydbver ssh $ mysql mysql> select version(); |
You are not limited to using full version specification. To use the latest matching version, reduce 5.6.43-rel84.3 down to 5.6.43 or even 5.6. To run other databases, replace percona-server with:
- pxc: Percona XtraDB Cluster
- mysql: Oracle MySQL Community Server
- mariadb: MariaDB
- mariadb-cluster: MariaDB Galera Cluster
- pg: Postgres build from https://www.postgresql.org/
- ppg: Percona Distribution for PostgreSQL
- psmdb: Percona Server for MongoDB
Multiple Instances
Several containers are not consuming a significant amount of resources. Actually five MySQL container instances will consume the same resources as five individual processes running on the same host. The syntax is:
1 | $ ./anydbver deploy <default node definition> node1 <node1 definition> node2 <node2 definition> ... |
For example, run two independent MySQL instances:
$ ./anydbver mysql node1 mysql
SSH access to default node:
1 2 3 | $ ./anydbver ssh # or $ ./anydbver ssh default |
All other nodes (replace node1 with other node names):
1 | $ ./anydbver ssh node1 |
Hostnames
You may have a server hostname specified. For example let’s run two Percona Server for MySQL instances:
1 2 3 4 5 | $ ./anydbver deploy ps:5.7 hostname:leader.percona.local node1 ps:5.7 hostname:follower.percona.local $ ./anydbver ssh leader or ./anydbver ssh leader.percona or leader.percona.local [root@leader ~]# mysql --host follower.percona.local --prompt '\h mysql>' follower.percona.local mysql> |
Replication
The most interesting part of modern open-source databases is replication. Even active-active replication setups are starting from a single server (leader or master/primary). Start the first node normally and attach additional nodes with master:nodename or leader:nodename. PXC or Galera servers could participate in both synchronous and asynchronous replication. Thus, for Galera clusters, you need galera-master or galera-leader syntax.
Start a 3 node Percona XtraDB cluster (latest 5.7):
1 | ./anydbver deploy pxc:5.7 node1 pxc:5.7 galera-master:default node2 pxc:5.7 galera-master:default |
Run master and two async slaves with Percona Server for MySQL and add all three servers to ProxySQL setup:
1 | ./anydbver deploy ps:5.7 node1 ps:5.7 master:default node2 ps:5.7 master:default node3 proxysql master:default |
Setup physical replication with slots for Postgresql 12.3:
1 | ./anydbver deploy pg:12.3 node1 pg:12.3 master:default |
Make a Mongo replica set named rs0:
1 | ./anydbver deploy psmdb replica-set:rs0 node1 psmdb master:default replica-set:rs0 node2 psmdb master:default replica-set:rs0 |
MongoDB Sharding
MongoDB sharding setup requires several server types: servers with data (shardsrv), configuration servers (configsrv), and mongos server:
1 2 3 4 5 6 7 8 | ./anydbver deploy \ psmdb:4.2 replica-set:rs0 shardsrv \ node1 psmdb:4.2 master:default replica-set:rs0 shardsrv \ node2 psmdb:4.2 master:default replica-set:rs0 shardsrv \ node3 psmdb:4.2 configsrv replica-set:cfg0 \ node4 psmdb:4.2 configsrv replica-set:cfg0 master:node3 \ node5 psmdb:4.2 configsrv replica-set:cfg0 master:node3 \ node6 psmdb:4.2 mongos-cfg:cfg0/node3,node4,node5 mongos-shard:rs0/default,node1,node2 |
Containers and Orchestration
The fact that we are already using containers (LXD) shouldn’t confuse you. We can still run docker images inside our nodes. Nested containers and Podman makes it possible.
Run Percona Monitoring and Management Docker containers
Let’s deploy the default node with Podman and run the Percona Monitoring and Management (PMM) server docker container in it. Percona Server for MySQL 5.7 with PMM client will run on node1:
1 | ./anydbver deploy pmm node1 ps:5.7 pmm-client pmm-server:default |
Run multi-node Kubernetes cluster
Kubernetes also could utilize nested containers. There are several small Kubernetes distributions: minikube, microk8s, k3s, k0s. The simplest fully functional Kubernetes could be implemented with k3s.
The first k3s LXD container executes the API server and workers. Additional LXD containers could run more workers. Multiple workers are important to run complex HA applications with hostname anti-affinity.
1 2 | ./anydbver deploy k3s node1 k3s-master:default node2 k3s-master:default ./anydbver ssh |
On a default node, we can execute kubectl or helm:
1 2 3 4 5 | $ kubectl get nodes NAME STATUS ROLES AGE VERSION ihanick-node2 Ready <none> 14m v1.19.2+k3s1 ihanick-default Ready master 17m v1.19.2+k3s1 ihanick-node1 Ready <none> 15m v1.19.2+k3s1 |
Percona Kubernetes Operator for Percona XtraDB Cluster
The Kubernetes cluster allows running stateful applications, like databases, services (similar to Amazon AWS S3), or monitoring solutions. Let’s start:
- 4 node cluster
- MinIO (implements S3 api) for backups
- Percona XtraDB Cluster (managed by the operator)
- Percona Monitoring and Management
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 | ./anydbver deploy k3s \ node1 k3s-master:default \ node2 k3s-master:default \ node3 k3s-master:default \ default k8s-minio k8s-pmm k8s-pxc backup ./anydbver ssh kubectl get pods NAME READY STATUS RESTARTS AGE svclb-monitoring-service-n5nsx 1/1 Running 0 20m svclb-monitoring-service-htssw 1/1 Running 0 20m svclb-monitoring-service-n9kt4 1/1 Running 0 20m svclb-monitoring-service-7btbh 1/1 Running 0 20m minio-service-6db6667fb9-tk69n 1/1 Running 0 20m monitoring-0 1/1 Running 0 20m percona-xtradb-cluster-operator-7886ccf6b5-rtwxc 1/1 Running 0 18m cluster1-pxc-0 2/2 Running 2 17m cluster1-haproxy-0 2/2 Running 0 17m cluster1-haproxy-1 2/2 Running 0 12m cluster1-haproxy-2 2/2 Running 0 12m cluster1-pxc-1 2/2 Running 1 13m cluster1-pxc-2 2/2 Running 1 10m |
Percona Kubernetes Operator for Percona Server for MongoDB
You can install and configure the database in the same way, regardless of the architecture:
1 2 3 4 5 | ./anydbver deploy k3s \ node1 k3s-master:default \ node2 k3s-master:default \ node3 k3s-master:default \ default k8s-minio k8s-mongo backup |
Zalando Postgres Operator
You are not limited to Percona-only products and can run other K8s operators:
1 | ./anydbver deploy k3s node1 k3s-master:default node2 k3s-master:default node3 k3s-master:default default k8s-pg |
Summary
AnyDbVer is a useful deployment tool for experiments with:
- MySQL/MariaDB/Percona Server for MySQL/Postgresql/MongoDB
- MySQL replication: asynchronous replication, Percona XtraDB Cluster, MariaDB Galera Cluster, InnoDB Cluster
- MongoDB replication and sharding
- Proxysql
- Kubernetes, including Percona Kubernetes Operator for Percona Server for MongoDB, Percona Kubernetes Operator for Percona XtraDB Cluster, Zalando Postgres Operator
- Percona Monitoring and Management (PMM)
- Specific database versions for bug reproduction and bisection
AnyDbVer restrictions:
- It is not a production deployment tool.
- The deployment process takes a long time. The tool downloads and installs all packages from OS and Vendor repositories.
