2

I have a "hot" MongoDB(4.0.5) capped collection with a timestamp (and several other) indexes. Many applications (Java, NodeJS & C++) insert (never update) records into this collection at a rate of ~50 inserts/second. Normally these inserts are very quick and take only a few milliseconds.

When I run a map-reduce query on the most recent 5 minutes of data, calls to insert new records into the source collection are blocked or delayed. New inserts during the map-reduce call can be delayed > 10 seconds which is roughly the time that it takes to run the map-reduce operation. The output from the map-reduce operation is written to the same database.

What can I do to tune MongoDB or the map-reduce operation so that inserts can continue to flow into the capped collection without delay while the map-reduce operation runs? I need to get the summary analysis in near real-time so I don't want to wait until the insert load is reduced before running the analysis.

I do not believe that this is a system level resource constraint. I have monitored CPU, network, i/o and memory during the map-reduce call and CPU idle time remains > 50% and network and i/o latency remains low. IO/wait times remain under 5%.

0

Maybe setting the nonAtomic: true flag could help.

Or you can try the Map-Reduce alternatives:

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.