Recently I wanted to play with the latest and greatest Percona Distribution for MongoDB Operator which had a bug fix I was interested in. The bug fix was merged in the main branch of the git repository, but no version of the Operator that includes this fix was released yet. I started the Operator by cloning the main branch, but the bug was still reproducible. The reason was simple – the main branch had the last released version of the Operator in bundle.yaml, instead of the main branch build:
1 2 3 4 | spec: containers: - name: percona-server-mongodb-operator image: percona/percona-server-mongodb-operator:1.9.0 |
instead of
1 2 3 4 | spec: containers: - name: percona-server-mongodb-operator image: perconalab/percona-server-mongodb-operator:main |
Then I decided to dig deeper to see how hard it is to do a small change in the Operator code and test it.
This blog post is a beginner contributor guide where I tried to follow our CONTRIBUTING.md and Building and testing the Operator manual to build and test Percona Distribution for MongoDB Operator.
Requirements
The requirements section was the first blocker for me as I’m used to running Ubuntu, but examples that we have are for CentOS and MacOS. For all Ubuntu fans below are the instructions:
1 2 3 4 5 6 7 | echo "deb [signed-by=/usr/share/keyrings/cloud.google.gpg] https://packages.cloud.google.com/apt cloud-sdk main" | sudo tee -a /etc/apt/sources.list.d/google-cloud-sdk.list curl https://packages.cloud.google.com/apt/doc/apt-key.gpg | sudo apt-key --keyring /usr/share/keyrings/cloud.google.gpg add - sudo apt-get update sudo apt-get install -y google-cloud-sdk docker.io kubectl jq sudo snap install helm --classic sudo snap install yq --channel=v3/stable curl -s -L https://github.com/openshift/origin/releases/download/v3.11.0/openshift-origin-client-tools-v3.11.0-0cbc58b-linux-64bit.tar.gz | sudo tar -C /usr/bin --strip-components 1 --wildcards -zxvpf - '*/oc' |
I have also prepared a Pull Request to fix our docs and drafted a cloud-init file to simplify environment provisioning.
Build
Get the code from GitHub main branch:
git clone https://github.com/percona/percona-server-mongodb-operator
Change some code. Now it is time to build the Operator image and push it to the registry. DockerHub is a nice choice for beginners as it does not require any installation or configuration, but for keeping it local you might want to install your own registry. See Docker Registry, Harbor, Trow.
./e2e-tests/build command builds the image and pushes it to the registry which you specify in IMAGE environment variable like this:
1 | export IMAGE=bob/my_repository_for_test_images:K8SPSMDB-372-fix-feature-X |
Fixing the Issues
For me the execution of the build command failed for multiple reasons:
1. Most probably you need to run it as root to get access to docker unix socket or just add the user to the docker group:
1 | Got permission denied while trying to connect to the Docker daemon socket at unix:///var/run/docker.sock |
2. Once I ran it with root I got the following error:
1 | "--squash" is only supported on a Docker daemon with experimental features enabled |
It is quite easy to fix it by adding the experimental flag into /etc/docker/daemon.json file:
1 2 3 | { "experimental": true } |
I have added it into the cloud-init file and will fix it in the same PR in the docs.
3. The third failure was on the last stage of pushing the image:
1 | denied: requested access to the resource is denied |
Obviously, you should be authorized to push to the registry. docker login fixed it for me just fine.
Finally, the image is built and pushed to the registry:
1 2 3 4 | The push refers to repository [docker.io/bob/my_repository_for_test_images] 0014bf17d462: Pushed ... K8SPSMDB-372-fix-feature-X: digest: sha256:458066396fdd6ac358bcd78ed4d8f5279ff0295223f1d7fbec0e6d429c01fb16 size: 949 |
Test
./e2e-tests/run command executes the tests in e2e-tests folder one-by-one, as you see there are multiple scenarios:
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 | "$dir/init-deploy/run" || fail "init-deploy" "$dir/limits/run" || fail "limits" "$dir/scaling/run" || fail "scaling" "$dir/monitoring/run" || fail "monitoring" "$dir/monitoring-2-0/run" || fail "monitoring-2-0" "$dir/liveness/run" || fail "liveness" "$dir/one-pod/run" || fail "one-pod" "$dir/service-per-pod/run" || fail "service-per-pod" "$dir/arbiter/run" || fail "arbiter" "$dir/demand-backup/run" || fail "demand-backup" "$dir/demand-backup-sharded/run" || fail "demand-backup-sharded" "$dir/scheduled-backup/run" || fail "scheduled-backup" "$dir/security-context/run" || fail "security-context" "$dir/storage/run" || fail "storage" "$dir/self-healing/run" || fail "self-healing" "$dir/self-healing-chaos/run" || fail "self-healing-chaos" "$dir/operator-self-healing/run" || fail "operator-self-healing" "$dir/operator-self-healing-chaos/run" || fail "operator-self-healing-chaos" "$dir/smart-update/run" || fail "smart-update" "$dir/version-service/run" || fail "version-service" "$dir/users/run" || fail "users" "$dir/rs-shard-migration/run" || fail "rs-shard-migration" "$dir/data-sharded/run" || fail "data-sharded" "$dir/upgrade/run" || fail "upgrade" "$dir/upgrade-sharded/run" || fail "upgrade-sharded" "$dir/upgrade-consistency/run" || fail "upgrade-consistency" "$dir/pitr/run" || fail "pitr" "$dir/pitr-sharded/run" || fail "pitr-sharded" |
Obviously, it is possible to run the tests one by one.
It is required to have kubectl configured and pointing to the working Kubernetes cluster. If something is missing or not working the tests are going to tell you that.
The only issue I faced is the readability of the test results. The logging of the test execution is pretty verbose, so I would recommend redirecting the output to some file for further debugging purposes.
1 | `./e2e-tests/run >> /tmp/e2e-tests.out 2>&1 |
We in Percona rely on Jenkins to automatically test and verify the results for each Pull Request.
Conclusion
Contribution guides are written for developers by developers, so they often have some gaps or unclear instructions which sometimes require experience to resolve. Such minor issues might scare off potential contributors and as a result, the project does not get the Pull Request with an awesome implementation of the brightest idea. Percona embraces open source culture and values contributors by providing simple tools to develop and test the ideas.
Writing this blog post resulted in two Pull Requests:
There is always room for improvement and a time to find a better way. Please let us know if you face any issues with contributing your ideas to Percona products. You can do that on the Community Forum or JIRA. Read more about contribution guidelines for Percona Distribution for MongoDB Operator in CONTRIBUTING.md.
Percona Distribution for MongoDB is a freely available MongoDB database alternative, giving you a single solution that combines the best and most important enterprise components from the open source community, designed and tested to work together.
Download Percona Distribution for MongoDB Today!
