I have database backups in xtrabackup format. I would like to load them into hive without restoring to a mysql instance and use sqoop to then do the transfer.

If there are libraries to work with the format I can probably write my own utility to load the data into hive. I'm also more than happy to receive other suggestions on how to do that.

  • You'd go to all the trouble of writing something to do that when all you have to do is spin up an instance of MySQL? Sounds interesting, but unless you are a student doing this out of interest, I fail to see what benefit you could derive from such a strategy. The time you'd spend doing this (debugging - plus who's going to provide support?). – Vérace Mar 24 '16 at 17:55
  • The benefit is speed and correctness. Sqoop is really the worst. – Marcin Mar 24 '16 at 18:50

There are tools to extract data from XtraBackup data files. But don't forget about a redo log. Without it the Xtrabackup copy is incomplete. To make the backup usable you have to apply the redo log which means you need to restore the backup.

I would look at mydumper or mysqldump with -T option and then upload the dumps to hive.

  • do you have a linke to those tools? There are tools to extract data from XtraBackup data files. Thanks – Ivanov Aug 20 '18 at 7:10

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.