Comments on: MySQL Distributed Logical Backups: a Proof of Concept https://www.percona.com/blog/mysql-distributed-logical-backups-a-proof-of-concept/ Wed, 15 Jan 2020 12:19:02 +0000 hourly 1 https://wordpress.org/?v=6.5.2 By: Buchan https://www.percona.com/blog/mysql-distributed-logical-backups-a-proof-of-concept/#comment-10971681 Wed, 15 Jan 2020 12:19:02 +0000 https://www.percona.com/blog/?p=64381#comment-10971681 What you are really doing here is comparing a single serialized dump of all tables, to a parallel (n=3) dump of tables.

Could you test with mydumper ( https://github.com/maxbube/mydumper ), in order to differentiate between scaling via multiple connections dumping vs multiple servers?

Also, in my experience, parallelism in the dump part is less than half the problem, as restore from mysqldump typically scales much worse than creating the mysqldump.

Finally, the other problem that can occur is when, even though you have 100 tables, inevitably there will be one (or a few) table(s) that make up large percentage of the entire database size (e.g. an audit trail table or similar). mydumper seems to have features to help in this case, e.g. by chunking the tables into parts.

]]>
By: Eric https://www.percona.com/blog/mysql-distributed-logical-backups-a-proof-of-concept/#comment-10971677 Sat, 11 Jan 2020 15:03:48 +0000 https://www.percona.com/blog/?p=64381#comment-10971677 Hi,
interesting as approach to split the load, operation impact of “logical” backup done over multiple instance.

My point-of-view:
interesting
– when you don’t have advance storage capabilities, want to limit operational task impact on your production
– by using standard tool, architecture

But:
– doesn’t it make more complex ?
– implementation ?
– if you need to restore our master (user issue –> delete data –> replicated to other slave), you will need to restore your master and rebuild your replicated (DR more complex, RTO impacted)
– what is the cost on running multiple ‘host” and additional storage for (in this scenario) 3 replicas ?
– as you mention there is disk/Storage solution, certainly on modern one, they have advance feature can provide a other magnitude of service and in this context:
– snapshot : for TB, it takes seconds (only delta)
– clone : for TB, it takes seconds to have a clone of the running (real-time) master MySQL instance
– time efficiency: very fast
– space efficiency : for the read –> access the original block of the Master databases (shared resource). With your database of e.g. 1 TB you could have a clone with some MB/GB. You consume the same storage resource (cpu, disk, …) and to limit this you could do the clone from a older snapshot/backup.

]]>
By: Vaibhav https://www.percona.com/blog/mysql-distributed-logical-backups-a-proof-of-concept/#comment-10971660 Thu, 09 Jan 2020 20:09:32 +0000 https://www.percona.com/blog/?p=64381#comment-10971660 Simple and great idea.

Just to add few things as a suggestion ..

events and routines
to dump events , procedures and functions

also it would be great to have a hostname as part of each backup.

and last it would be great if there is a check for replication filters before execution of backup.

]]>
By: Daniel Guzmán Burgos https://www.percona.com/blog/mysql-distributed-logical-backups-a-proof-of-concept/#comment-10971659 Thu, 09 Jan 2020 18:53:00 +0000 https://www.percona.com/blog/?p=64381#comment-10971659 Hi Brad, it doesn’t. The backup command have an ampersand (&) at the end, which means that the execution of that command will continue in a async way. Unlock happens within the second.

]]>
By: Brad Mickel https://www.percona.com/blog/mysql-distributed-logical-backups-a-proof-of-concept/#comment-10971658 Thu, 09 Jan 2020 18:44:06 +0000 https://www.percona.com/blog/?p=64381#comment-10971658 I really like this idea, but why do you wait until the backup is complete to unlock the binlog on the master? Couldn’t that be done after issuing the “START SLAVE UNTIL” commands? This way it allows the master to commit again, but since the slaves are stopped their backups would still be consistent.

]]>
By: Andy Moore https://www.percona.com/blog/mysql-distributed-logical-backups-a-proof-of-concept/#comment-10971657 Thu, 09 Jan 2020 18:27:12 +0000 https://www.percona.com/blog/?p=64381#comment-10971657 Outside the box thinking from our friends at Percona ??

]]>