Mongo is using 5-7% of CPU, so why is my computer so slow? [closed]

Question

I've installed MongoDB (not sharded, single location) on an AWS EC2 instance, and I'm finding that when I'm running large insert operations via pymongo, my entire EC2 instance gets very sluggish. Yet when I check CPU usage, it's reasonably low. Why is this and is there any way to contain it? Thanks for any advice.

Answer 1

1. Use 'mongostat' to monitor the amount of inserts

2. Use 'iostat -x 1' to see how much of the disk limitation is used. 'top'/'htop' to see cpy/ram limitations. 'nethogs' for network limits.

3. If the disk limit isn't reached : Let your python script run from different locations (servers) at the same time, and see if you can increase the amount of inserts. The pymongo driver might have a limitation per connection also.

Or you can use the mongoclient as extra threads. But I found the mongo client even slower inserting than pymongo on my tests:

$ mongo testDB --eval "for( var i = 0; i < 50000; i++ ) { db.test.insert({x:i,y:'hi'}); }" &

$ mongo testDB --eval "for( var i = 0; i < 50000; i++ ) { db.test.insert({x:i,y:'hi'}); }" &

Python speed insert test:

import sys
import datetime

from pymongo import MongoClient
connection = MongoClient("mongodb://10.x.x.x", w=0)

db=connection.testing
test = db.test

tstart = datetime.datetime.now()
d = {'counter' : 0}
while d['counter'] < 1000000:
  d['counter'] += 1
  testjs = {"x":d['counter'], "y":"hi" }
  try:
    test.insert(testjs)
  except:
    print "Unexpected error:", sys.exc_info()[0]

tend = datetime.datetime.now()
tdiff = tend - tstart
print tdiff.total_seconds(), ' ',tdiff.microseconds

Todo extra in the python script:

• connect to mongos with a sleep between the inserts to see what happens when a primary of a shard goes down
• insert more than "hi" to test splitting of chunks
• ...