InnoDB + SSD ==> no fragmenatation worth noting ==> Don't use OPTIMIZE TABLE
. What you claim to experience in this area does not make sense.
innodb_buffer_pool_size
should be about 70% of the RAM after accounting for your other apps. (1G is too low.) This may be part of the performance issue you are seeing.
OPTIMIZE TABLE
==> lots of extra writes ==> wear out SSDs without wear-laveling.
This may help: innodb_doublewrite = OFF
. It poses a slight risk of data corruption, depending on the details of the OS and SSD.
This may help: sync_binlog = OFF
Are you using autocommit = ON
? Or regularly using BEGIN...COMMIT
? I suggest using the latter around the various inserts/updates for each page being processed. (I assume that involves several writes to several tables.)
If you have some spinning drive space handy, consider moving "logs" to there. (The old wisdom was that the 'sequential' nature of logs was well-suited to HDD. I don't know if it is still true now, but it would save some of the "wear" of the SSDs.)
Turn off the Query Cache.
For further analysis: http://mysql.rjweb.org/doc.php/mysql_analysis
Higher QPS
For the revised QPS:
- Use "enterprise" SSDs
- Set buffer_pool as big as possible without swapping
- do the BEGIN..COMMIT (where practical)
- look for any "slow" queries during the spikes. (During a spike, the system is at risk of getting 'stuck' due to the rising number of connections that seem to never terminate. Speeding up a long-running query may be a quick fix. Look for such now as a prophylactic.