1

I run a task in my Python application of ~100K jobs which takes around 5 hours, since the beginning my in RDS instance CPU starts growing...

Instance type: Postgresql 9.5 | db.t2.large

enter image description here

I have the following performance counters:

CPU 68.1%
Memory 7,250 MB
Storage 4,010 MB
Read IOPS 0.55/sec
Write IOPS 27.2/sec
Swap Usage 0 MB

Settings:

max_connections {DBInstanceClassMemory/12582880}
pg_stat_statements.track ALL
shared_preload_libraries  pg_stat_statements
tcp_keepalives_count  1
tcp_keepalives_idle   120
tcp_keepalives_interval 60
temp_buffers  {DBInstanceClassMemory/512}
track_activity_query_size 2048

I'm using pganalyze to monitor database.

Any pointers how to debug CPU increase? All 100K job consist of same queries (1 inserts/4 updates per job).

3
  • 2
    Just a guess but PG mvcc implementation can cause more writes to disk, if you do many updates, take a look at the dead tuples statistics on your tables - it might be very large causing a load on the storage layer which manifest as cpu io-waits (which show as cpu load in monitors)
    – cohenjo
    Oct 3 '16 at 5:29
  • Does the CPU utilization cause other problems?
    – dezso
    Oct 4 '16 at 7:35
  • Yes, if I have more than 100K queries, DB response takes very long time > 1000 ms and application start acting weird. Seems to be a problem with my UPDATE statements which are looking for a non-indexed field. Will add Index and verify
    – gogasca
    Oct 4 '16 at 8:39

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Browse other questions tagged or ask your own question.