Take the 2-minute tour ×
Database Administrators Stack Exchange is a question and answer site for database professionals who wish to improve their database skills and learn from others in the community. It's 100% free, no registration required.

My collection with a _id and a single field index is run on a single node/on localhost, has 48GB RAM and 16 core CPU available. The hdd is in RAID1, without SSD cache. Mongo version 2.4.

I run a mongoimport with 5Mio documents dump with a total of 5GB with mongoimport. It starts fast and slows down very quickly.

This starts fast with about 11000/second. After 3 minutes going down to 9000/s, again 1 minute later it is 7000/s and after some more minutes I we are in slow motion on 200/s. The mongoimport takes for hours instead of some minutes.

mongostat shows increasing LOCKs and faults:

insert  query update delete getmore command flushes mapped  vsize    res faults  locked db idx miss %     qr|qw   ar|aw  netIn netOut  conn       time 
*0     *0     *0     *0       0     1|0       0   912g  1825g    21g      1 tdb:74.0%          0       0|0     0|1     5m     3k    10   03:20:21 
...
79     *0     *0     *0       0     1|0       0   912g  1825g    21g     10 tdb:33.6%          0       0|0     0|0    47k     3k    10   03:22:22 
...
104     *0     *0     *0       0     1|0       0   912g  1825g    21g     13  tdb:7.9%          0       0|1     0|1    60k     3k    10   03:22:33 
...
86     *0     *0     *0       0     1|0       0   912g  1825g    21g     11  tdb:0.8%          0       0|0     0|1    52k     3k    10   03:23:17 
...
128     *0     *0     *0       0     1|0       0   912g  1825g    21g      9 tdb:58.3%          0       0|0     0|1    76k     3k    10   03:25:00 
...
95     *0     *0     *0       0     1|0       0   912g  1825g    21g     11 tdb:39.3%          0       0|0     0|0    58k     3k    10   03:26:07 
...
86     *0     *0     *0       0     1|0       0   912g  1825g    21g     12 tdb:88.8%          0       0|0     0|1    51k     3k    10   03:27:12 

I run mongodb without --upsert and also tried with only the _id index. Furthermore reIndex()did not improve this.

How to improve this and remain the speed of bulk INSERTs constantly?

OS: Debian 6, 64Bit

MongoDB: 2.4.4

share|improve this question
    
Couple of things missing here - what exact version and what OS? There was a significant performance regression on Windows in earlier versions of 2.4 (fixed in 2.4.4 - see jira.mongodb.org/browse/SERVER-9242) –  Adam C Jul 12 '13 at 9:59
    
added OS, MongoDB version. Furthermore, upgraded RAM to 48GB. without improvemenent. –  ledy Jul 12 '13 at 13:09
1  
Majority of your RAM is not being used - 21g out of 48g even though you're constantly page faulting according to mongostat. What's readahead set at? Anything else running on this system using up RAM? –  Asya Kamsky Jul 27 '13 at 2:11

Your Answer

 
discard

By posting your answer, you agree to the privacy policy and terms of service.

Browse other questions tagged or ask your own question.