I have a table with 12 million of rows and a following task I should frequently perform:
- Get search results from somewhere (50 rows). Each result has a key looking as a md5 hash and a table's PM is built on this field.
- Check what rows are currently stored.
- Store all other rows
Question is what is the best way to perform steps 2-3. I use PHP and Doctrine so not all tricky queries are possible to use. For example I can't use bulk inserts so need to run INSERT up to 50 times in a row. I see two possible ways:
- run SELECT ... WHERE id IN(...) with all 50 IDs and see what is returned, then run as many inserts as I need
- run 50 inserts and catch duplicated ID error