Take the 2-minute tour ×
Database Administrators Stack Exchange is a question and answer site for database professionals who wish to improve their database skills and learn from others in the community. It's 100% free, no registration required.

I want to import a Wikipedia XML dump file into MySQL. The uncompressed XML dump file is 65GB. Does this mean I should free up almost 64GB in the MySQL data path? Is there any way to estimate the required space?

Here is the sql schema:https://git.wikimedia.org/blob/mediawiki%2Fcore.git/HEAD/maintenance%2Ftables.sql

share|improve this question
add comment

1 Answer

up vote 1 down vote accepted

I have dealt with questions like this in both directions

Your question is a little different because you did not give anything about the target MySQL Instance (what Storage Engine will be used, which columns are TEXT/BLOB, what indexing is needed for your data, etc.)

IMHO the most straightforward way I can think of is the following

METHOD #1

Write some Perl/Python script to sum the length of all XML tag delimiters. Subtract the sum you get from the length of the file. That would essentially be the size of a CSV file less delimiters. This would be the bytes of the raw data.

METHOD #2

Do METHOD #1 and also take the count of all XML tag delimiters and add that count to the result of METHOD #1. That would like adding commas back into a CSV file. This make be closer in size to an actual CSV should you print the output to such a file.

share|improve this answer
    
If that schema was the result of converting the XML, simply remove all the comments. I would estimate the data pages only from the dump size you get. Indexes are built at the end of each tables load. If your indexes are properly designed to accommodate all your queries, then add another 20% of that figure. –  RolandoMySQLDBA Jun 20 '13 at 14:24
add comment

Your Answer

 
discard

By posting your answer, you agree to the privacy policy and terms of service.

Not the answer you're looking for? Browse other questions tagged or ask your own question.