We have three mongo shards with db version v3.2.10.The primary shard is running with very high CPU usage ~675% while other 2 shard node are running ~12% CPU usage.
Here is output for $top command of primary shard:
PID USER PR NI VIRT RES SHR S %CPU %MEM TIME+ COMMAND
3825494 mongod 20 0 44.4g 37g 10m S 662.5 62.2 14276:02 mongod
2803 root 20 0 4380 84 0 S 0.3 0.0 71:59.57 rngd
output of $top command for secondary shard:
PID USER PR NI VIRT RES SHR S %CPU %MEM TIME+ COMMAND
2504833 mongod 20 0 38.3g 36g 9m S 10.0 60.9 1686:11 mongod
3891605 qateam 20 0 454m 12m 0 S 2.0 0.0 53:34.66 node_exporte
Output of db.currentOp() at shard1 where CPU is high and with more secs_running
1) { "desc" : "rsSync", "threadId" : "140545421088512", "active" : true, "opid" : 76313, "secs_running" : 329997, "microsecs_running" : NumberLong("329997131632"), "op" : "none", "ns" : "local.replset.minvalid", "query" : { }, "numYields" : 0, "locks" : { }, "waitingForLock" : false, "lockStats" : { "Global" : { "acquireCount" : { "r" : NumberLong(72414875), "w" : NumberLong(54311146), "R" : NumberLong(18103716), "W" : NumberLong(18103716) }, "acquireWaitCount" : { "R" : NumberLong(132), "W" : NumberLong(3942872) }, "timeAcquiringMicros" : { "R" : NumberLong(18610043), "W" : NumberLong(1490223084) } }, "Database" : { "acquireCount" : { "r" : NumberLong(6), "w" : NumberLong(1), "W" : NumberLong(54311145) }, "acquireWaitCount" : { "W" : NumberLong(1) }, "timeAcquiringMicros" : { "W" : NumberLong(61) } }, "Collection" : { "acquireCount" : { "r" : NumberLong(5) } }, "Metadata" : { "acquireCount" : { "w" : NumberLong(1) } }, "oplog" : { "acquireCount" : { "r" : NumberLong(1), "w" : NumberLong(1) } } } } 2) { "desc" : "WT RecordStoreThread: local.oplog.rs", "threadId" : "140545378072320", "active" : true, "opid" : 715968754, "secs_running" : 1310, "microsecs_running" : NumberLong(1310975777), "op" : "none", "ns" : "local.oplog.rs", "query" : { }, "numYields" : 0, "locks" : { }, "waitingForLock" : false, "lockStats" : { "Global" : { "acquireCount" : { "r" : NumberLong(1), "w" : NumberLong(1) } }, "Database" : { "acquireCount" : { "w" : NumberLong(1) } }, "oplog" : { "acquireCount" : { "w" : NumberLong(1) } } } }
How can I control and keep it down to normal CPU usage range.