I would look to the InnoDB redo log file size. 

The symptoms you describe is typical if you fill up the redo log with changes, which forces a "synchronous flush" — MySQL blocks further changes until it can free up a portion of the redo log by flushing dirty pages from the buffer pool.

RDS used to use an absurdly small redo log file size by default, 128M if I recall. For years they did not allow changing the size. But in the last couple of years they do allow changing it.

Here's how to check the size of your redo log file in megabytes:

    mysql> SELECT @@innodb_log_file_size / 1024 / 1024;

To change it, I think you'd use the RDS parameter groups UI, then restart your RDS instance to apply the change.