1

We have recently upgraded to MongoDB 3.6 and since then we have noticed that the journal files keep building up and are not getting cleaned. Due to this, the disk usage keeps growing until there is no space left on the disk.

This issue does not occur with MongoDB 3.4. Is there anyway to enforce cleanup of journal files in MongoDB 3.6?

Update on data volumes:
We process around 148 GB of data every day. The disk space that the MongoDB data files use is around 60 GB. However, the journal files use up additional 70 GB of space everyday. Since the journal files are not getting cleaned up, the total disk space used increases by 70 GB everyday.

  • Hi Chandana, and welcome to dba.stackexchange. Can I ask for a clarification: what size of data are you talking about - e.g. amount per day in GB? – Vince Bowdren Mar 22 '18 at 16:04
  • Hello! Thanks for your response! We process around 148 GB of data every day. The disk space that the MongoDB data files use is around 60 GB. However, the journal files use up additional 70 GB of space everyday. Since the journal files are not getting cleaned up, the total disk space used increases by 70 GB everyday. – Chandana Mar 23 '18 at 7:40
  • What are your specific versions of MongoDB 3.6.x server and operating system? Also, what sort of deployment do you have (standalone, replica set, or sharded cluster)? – Stennie Mar 23 '18 at 10:24
  • 1
    We are using a standalone MongoDB 3.6.3 on a 64-bit Windows 7 machine. Also, the storage engine used is WiredTiger – Chandana Mar 23 '18 at 12:36
2

This is a known issue recently discovered in the MongoDB 3.6.3 release when using WiredTiger on Windows: SERVER-33982: Unbounded growth of pre-allocated log files.

This issue will be corrected in the 3.6.4 production release, so upgrading to MongoDB 3.6.4 (or newer) when available should resolve the issue.

In the interim, possible workarounds are:

  • Disable WiredTiger preallocation in 3.6.3

    • In your MongoDB config file:

      storage:
        wiredTiger:
          engineConfig:
            configString: log=(prealloc=false)
      
    • Or, via command-line parameter:

      --wiredTigerEngineConfigString=log=(prealloc=false)
      
  • Downgrade to MongoDB 3.6.2

  • Restart your mongod to free up the preallocated space
| improve this answer | |
  • Thanks for your response! I tried to disable WiredTiger preallocation, but I'm unable to start MongoDB after that. I tried both the config option as well as the command line option; and in both cases MongoDB did not start. Am I missing something? – Chandana Mar 26 '18 at 8:14
  • @Chandana Can you check the MongoDB log file to see if there were any specific error messages on startup? Typically there should be a reason noted if mongod fails to start (eg. invalid or unrecognised option) – Stennie Mar 26 '18 at 9:20
  • Here is an extract from the log file pertaining to this - 2018-03-26T16:20:54.872+0530 F CONTROL [thread51] Encountered an error in PeriodicRunnerASIO: The I/O operation has been aborted because of either a thread exit or an application request. 2018-03-26T16:20:54.872+0530 I FTDC [serviceShutdown] Shutting down full-time diagnostic data capture 2018-03-26T16:20:54.876+0530 I STORAGE [serviceShutdown] WiredTigerKVEngine shutting down 2018-03-26T16:20:55.047+0530 I STORAGE [serviceShutdown] shutdown: removing fs lock... 2018-03-26T16:20:55.047+0530 I CONTROL [serviceShutdown] – Chandana Mar 26 '18 at 10:53
  • In continuation to my above comment, I made some more trials and realized that the issue was with the single quote used to wrap log=(prealloc=false). When I tried the following command line argument, I was able to start MongoDB --> mongod --config "C:\Program Files\MongoDB\Server\mongod.conf" --auth --wiredTigerEngineConfigString=log=(prealloc=false) --install – Chandana Mar 26 '18 at 12:10
  • To summarize, the command-line alternative for the first workaround that was suggested worked well. However, setting configString in the config file has some issue (despite following the required indentation). Is there anything that I may have missed? – Chandana Mar 26 '18 at 15:21
0

This issue does not occur with MongoDB 3.4. Is there anyway to enforce cleanup of journal files in MongoDB 3.6?

As you can limit the journal files to adding the smallfiles=truein mongod.conf configuration file. As per @mnemosyn here and MongoDB BOL here.

For further your ref here, here and here

| improve this answer | |
  • FYI, the smallfiles configuration option is specific to the MMAPv1 storage engine. – Stennie Mar 23 '18 at 23:33
  • @Stennie, Yes stennie "smallfiles=true" is applicable in case of MMAPv1 storage engine. – Md Haidar Ali Khan Mar 27 '18 at 5:31

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.