The conventional approach is to set up multiple replicated slaves.
You then backup from these slaves periodically (mysqldump
is probably fine for this part still).
The master need not be blocked while the slaves are being backed-up; it will simply continue to replicate once those backups are done. Stagger the backups so you always have an up-to-date slave.
Replication is not CPU-expensive (unless you have a crazy level of database activity), and the slaves' backup process won't impact the master at all.
And everything should have redundant storage.
Anecdote: At a previous job we had a 600GB database that was backed up in the same way that you're doing it, but over a 10Mbit pipe to the target, and it would block the entire database for 11 hours every night! Eesh. Never did get replication in place there (lol) but upgrading the networking to 100Mbit meant it could be done in 2 hours or so instead, which was "fine". Of course, with the database locked during the backup (which it should be!), and thus the actual service unavailable for that duration, a performance hit from the backup process was not even observable from the outside. In your case you have multiple distinct databases so the situation isn't quite the same — you don't have to lock everything for the whole time.