What kind of setup do I need to have to be able to insert 10M records/minute, and maintain/query tables of 300M rows?
My data is two varchar(50) fields that are indexed, and one is unique. No other columns.
My script is batching INSERTS, and I am able to get to ~1M records/min, which deteriorates after a while (at 100M added its down to half or a bit less - 400k/min or so).
Using COPY is not an option because it terminates every time there is a duplicate entry.
EDIT: hardware is: 96GB RAM, 32 cores, SSD hard drives.