1

I run a task in my Python application of ~100K jobs which takes around 5 hours, since the beginning my in RDS instance CPU starts growing...

Instance type: Postgresql 9.5 | db.t2.large

enter image description here

I have the following performance counters:

CPU 68.1%
Memory 7,250 MB
Storage 4,010 MB
Read IOPS 0.55/sec
Write IOPS 27.2/sec
Swap Usage 0 MB

Settings:

max_connections {DBInstanceClassMemory/12582880}
pg_stat_statements.track ALL
shared_preload_libraries  pg_stat_statements
tcp_keepalives_count  1
tcp_keepalives_idle   120
tcp_keepalives_interval 60
temp_buffers  {DBInstanceClassMemory/512}
track_activity_query_size 2048

I'm using pganalyze to monitor database.

Any pointers how to debug CPU increase? All 100K job consist of same queries (1 inserts/4 updates per job).

Improve this question
3
  • 2
    Just a guess but PG mvcc implementation can cause more writes to disk, if you do many updates, take a look at the dead tuples statistics on your tables - it might be very large causing a load on the storage layer which manifest as cpu io-waits (which show as cpu load in monitors)
    – cohenjo
    Oct 3, 2016 at 5:29
  • Does the CPU utilization cause other problems?
    – dezso
    Oct 4, 2016 at 7:35
  • Yes, if I have more than 100K queries, DB response takes very long time > 1000 ms and application start acting weird. Seems to be a problem with my UPDATE statements which are looking for a non-indexed field. Will add Index and verify
    – gogasca
    Oct 4, 2016 at 8:39

0

Reset to default

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge that you have read and understand our privacy policy and code of conduct.