0

I know that with NDB you can not do large transactions, if you do data nodes can restart into read-only mode and corrupt table.

I need to copy records into a new table but the thing is this table contains millions of rows in staging, the live server has even 10 times that amount.

how can I divide my copy into smaller transactions so I will not kill my server?

Improve this question
2
  • Will there be writes (delete/insert/update) against the table while the "copy" is being performed?
    – Rick James
    Commented Jan 14, 2022 at 21:52
  • Was it your choice to use NDB? I ask because that Engine has a rather narrow niche, and a casual user should not pick NDB.
    – Rick James
    Commented Jan 14, 2022 at 21:53

2 Answers 2

Reset to default
0

ndb_move_data sounds like what you are looking for.

Improve this answer
1
  • thank you, seems like exactly what I need (sorry, I can't upvote, not enough rep.)
    – Rait
    Commented Jan 17, 2022 at 10:23
0

If you are dealing with larger transactions you can increase MaxNoOfConcurrentOperations. Default is 32k so in a two node cluster it means that at one time there can be only ~64k rows involved in the transaction.

But also in NDB cluster it is recommended to use smaller transactions anyway. Simplest way usually is by using LIMIT clause.

 INSERT INTO prod_table SELECT * FROM staging_table LIMIT 10000;
Improve this answer

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.