We are migrating lot of data from our old database to new database.
Presently, both of them are actively[writing/reading] used based on flows/activity as migration is in progress.
Here's the real challenge for me now. We have old database table which's still getting used now.
Atleast 1,00,000 new records are getting inserted every day and we need to move 6months old data [01-01-2014] to new database.
We are using MySQL in both places. The new table to old table need some logical-transformation of data which will be carried through Java-JDBC, as the table structures and functionality is changed.
What are the ideal steps that I need to follow and taken care to do this transformation as quick as possible without fail.
As per my calculation, we will be migrating atleast 2,00,00,000 records.
Presently index exists only on ID column. creation_date exists without index How to fetch efficiently these records, a better query to fetch?? Efficient way to transform using multi-threading Efficient way to write the data into new database