Comments on: Optimizing PXC Xtrabackup State Snapshot Transfer https://www.percona.com/blog/optimizing-pxc-xtrabackup-state-snapshot-transfer/ Fri, 16 Jun 2023 19:12:07 +0000 hourly 1 https://wordpress.org/?v=6.5.2 By: Jay Janssen https://www.percona.com/blog/optimizing-pxc-xtrabackup-state-snapshot-transfer/#comment-10966023 Thu, 24 Mar 2016 15:50:56 +0000 https://www.percona.com/blog/?p=31326#comment-10966023 @john: My understanding of LVM snapshots (and I could be corrected) is when you take and hold them under the hood any block that changes must be copied and modified so the old block persists for the snapshot. This adds an IO write penalty while a snapshot is active.

Other filesystems like ZFS or SANs are generally always copy on write no matter what, so snapshots in those cases are much less expensive.

So, it’s not about the time to “get” the snapshot, it’s the extra IO while you hold it so you can copy that 2TB from it.

]]> By: john https://www.percona.com/blog/optimizing-pxc-xtrabackup-state-snapshot-transfer/#comment-10966022 Thu, 24 Mar 2016 15:45:25 +0000 https://www.percona.com/blog/?p=31326#comment-10966022 Jay, very helpful article. Our primary db is >2T and growing. SST is scary to say the least. Backup and restore using lvm seems like a good solution to us and we’re starting to explore it. I wonder what specific performance overhead you saw in your environment. My experience with lvm is the opposite; quick flush with read, do the lvm, done. Typically complete in single digit minutes. Am I missing something? As for being a manual effort to get the failed node back in sync, I believe for us that will always be the case.

]]> By: Jay Janssen https://www.percona.com/blog/optimizing-pxc-xtrabackup-state-snapshot-transfer/#comment-10928619 Mon, 24 Aug 2015 12:17:48 +0000 https://www.percona.com/blog/?p=31326#comment-10928619 @sebastian: It certainly is possible, though I’m not a huge fan of LVM snapshotting due to performance overhead. I’ve seen “manual SST” situations using snapshots in popular cloud providers block storage. (When I say “manual”, I mean the backups and new node provisioning is done by hand to avoid regular SST, it’s not fully scripted/automated). What you’re describing is fundamentally the same thing.

BTW, LOCK TABLES FOR BACKUP is supposed to block DDLs 😉

@fred: Indeed. I confess not exploring all the ways to accomplish this. I liked the compressor/decompressor settings because you can drop anything into that can handle unix pipes. I’ll play around with your settings a bit and see if I can get any further improvements.

]]> By: Frederic Descamps https://www.percona.com/blog/optimizing-pxc-xtrabackup-state-snapshot-transfer/#comment-10924948 Fri, 21 Aug 2015 10:54:23 +0000 https://www.percona.com/blog/?p=31326#comment-10924948 Hi Jay,

What about using xbstream and the xtrabackup settings ?

Like:

[sst]
streamfmt=xbstream

[xtrabackup]
compress
compact
parallel=16
compress-threads=16
rebuild-threads=16

]]> By: Sebastian https://www.percona.com/blog/optimizing-pxc-xtrabackup-state-snapshot-transfer/#comment-10924817 Fri, 21 Aug 2015 08:16:28 +0000 https://www.percona.com/blog/?p=31326#comment-10924817 Jay,

Thanks for this info.

I’m dealing with a setup that has a large number of tables and because each customer has a separate database, we’ve got a lot of DDL statements to create new schemas. SST is taking long due to number of tables/files and DDL statements are blocked (even with LOCK TABLES FOR BACKUP).
The solution here was to use LVM snapshots (rsync datadir from snapshot and start Joiner node with GTID from “grastate.dat”).

Do you think it would be possible to implement LVM snapshots for SST? It would be quite useful in cases like this.

Regards
Sebastian

]]>