Take the 2-minute tour ×
Database Administrators Stack Exchange is a question and answer site for database professionals who wish to improve their database skills and learn from others in the community. It's 100% free, no registration required.

I have a data file with 100 million documents. The number of documents is expected to always remain between 100 million and 200 million for atleast a year. I have 15 GB Ram. Currently the update operation takes 2 hours to load. Suppose I would like to increase the performance to require 40 minutes to load, will increasing RAM be required? How much RAM would I need to add in order to achieve that increase? Please do let me know if I must provide additional details.

I have indexes for the updated fields as well as I have set --nojournaling option. Currently I have optimized the update process by using syncdelay 0 and then manually fsync-ing every 100K records. Then I implemented a system with only in-place updates rather than updates that add new fields or increase document size. Now I feel that I am at a point where increasing RAM might be the only way to go.

share|improve this question
    
without knowing what is limiting your insert/update rate, it's hard to suggest whether it's RAM. It is likely that it is either RAM or disk IO bandwidth. Do the indexes fit in RAM? –  Asya Kamsky Jul 27 '13 at 2:05
    
@AsyaKamsky yeah the indexes fit in RAM. I heard that for optimal performance of MongoDB it is necessary that the entire working set must fit in RAM. Does 'working set' mean the indexes only? or does it mean the entire D B? Thank you very much –  Sai Jul 29 '13 at 14:15
    
@AsyaKamsky What do you mean by 'what is limiting your insert/update rate'? Do you mean that there is some kind of design flaw or do you mean that some kind of bg process that's blocking the RAM? –  Sai Jul 29 '13 at 14:18
add comment

Your Answer

 
discard

By posting your answer, you agree to the privacy policy and terms of service.

Browse other questions tagged or ask your own question.