5

I am building an index in MongoDB for a large collections (~300GB) and the db terminal window keeps printing:

[DataFileSync] flushing mmaps took 10920ms for 151 files

The time that it takes to flush mmaps is usually around 10 seconds and sometimes higher. I'm running MongoDB on a high performance machine so RAM availability is not an issue.

What does the data file sync message mean?

Is it an issue??

EDIT:

The real problem with the flushing routine is that MongoDB seems to freeze while doing that.

Even simple queries from a terminal window on the same host have to wait for the flushing to finish in order to complete and ~10 seconds (up to 150s) is way too long. The routine happens (at irregular intervals of 1, 2, or 3 mins) even if no operations are run on the server and there are zero connections active.

I'm running MongoDB 2.2.1 2008R2+ on Windows Server 2008 R2.

  • Apparently it is since you're posting a question. Are end-users complaining? Check the IO on your disk. I don't have any experience with Mongo, but I'd assume that you can configure these flushes. – swasheck Nov 13 '12 at 21:12
4

MongoDB flushes the data changes in memory to disk using a background thread every 60 seconds (it's tunable, see syncdelay). During normal operations, that is going to be inserts, updates, deletes and the like (including changes to indexes) and will roughly correlate to your activity in terms of those operations. The mmaps piece relates to the fact that MongoDB uses memory mapped files

When you build an index, you are going to be creating a lot of new in-memory data and that is all going to be persisted to disk every 60 seconds. How quick that flush is will be a function of:

  • How much data has to be flushed to disk (in your case how much index building has gone on)
  • How fast your disk can write that data out

Given that your example shows that it is taking 10 seconds to flush, that would imply that the disk is struggling a bit - single digit seconds is high, double digits is worrying. Of course it does depend on the nature of your disk - if an SSD were taking 10 seconds (for example) that would need to be a huge amount of data.

That explains the messaging, but besides all that index builds are intensive (and blocking) operations - you can run them in the background on a primary to avoid some of the impact, but they will run in the foreground on secondaries until SERVER-2771 is fixed. As a result the recommended way to build indexes with a replica set is outlined here:

http://docs.mongodb.org/manual/administration/indexes/#index-building-replica-sets

The impact will depend on how long the build takes, what you are doing at the time etc. but if you follow the instructions above you should be able to avoid any of those effects impacting your end users.

  • Adam, thanks a lot! I edited the question a bit adding some details about the problem (MongoDB freezes while flushing mmaps even if no indexes are being built). Do you know anything about this specific issue? Also, everything should be fine with the disk. Thanks! – Marsellus Wallace Nov 14 '12 at 15:15
  • Everything may be fine with the disk, but it may just not be fast enough - if it is freezing when flushing to disk, then that is likely the issue - the amount of data you are writing to disk every 60 seconds is taking too long to complete and that means that your disk is not quick enough for the data volume you are writing out. Then, any other operation that requires data from the disk is then going to have to block while waiting for it to complete, hence the freezing and slow down. – Adam C Nov 15 '12 at 1:46
  • downgraded to 2.0.8-rc0, this specific problem seems to be gone... (I still have others regarding the fact that MongoDB starts using 100% of RAM after a while but that's another story...) – Marsellus Wallace Nov 16 '12 at 15:42
  • 2.2 is more concurrent that 2.0, so it was likely doing more. By going back to 2.0 you've likely just reduced the load on your system a bit and not actually fixed the root cause. As for using 100% of RAM, that's what is supposed to happen, assuming you have more data than RAM and are running long enough - see mongodb.org/display/DOCS/Caching – Adam C Nov 16 '12 at 23:46
0

While starting a MongoDB service through mongod.exe, a few parameters can be specified, syncdelay is one of them.

I used the following command to start the service:

mongod.exe --syncdelay 10 --quiet --directoryperdb --bind_ip 127.0.0.1 --port 27018 --dbpath "D:\dbPath

This updates MMAP files every 10 seconds instead of the default 60 seconds.

In my case, lots of data are being written, so using this option could solve my problem.

I hope this will work for you as well.

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.