How to keep Mongodb log small via the conf file (or other persistent method)

Question

This is related to my question in stackoverflow where vb.net simply hang on updating a mongodb query.

https://stackoverflow.com/questions/12780258/insert-mongodb-using-vb-net/12796593#12796593

After long soul searching we check the log. The log is 3GB in size.

A log with that size is absolutely useless. I cannot have any text editor to open it.

How to keep the log of mongodb small, say 50 mb max.

We are only interested in the last few commmand anyway not the whole log.

Is there a setting for that in MongoDB? If so what settings:

Note: The question is simmilar with: https://stackoverflow.com/questions/8777449/mongodb-log-file-size

However, that question doesn't ask

How do I do so by changing some text files or something persistent so that the setting remain every time I start mongodb.

Answer 1

You might already know the answer just posting the answer here for someone else like me ending in this post :D

vi /etc/mongo.conf
smallfiles = true

This would set smallfiles true once you restart your mongodb.

UPDATE:

Rounding a little bit more the answer smallfiles option in mongo what does is reducing the initial and journal data files size, this is useful if your machine do not have to much space or if you are gonna have multiple databases with small data each.

I my personal opinion smallfiles do not increase performance but like mentioned before could help for this specific scenarios another example is with Docker if you are not using -v "path/to/local/drive:"/data/db" then is better to set smallfiles true in my case if i don't the container breaks because it can't allocate the default data file space.

To add smallfiles to the mongo.conf you can do:

echo "smallfiles = true" >> /etc/mongo.conf
service mongod restart

More detailed info here: http://docs.mongodb.org/manual/reference/configuration-options/#storage.mmapv1.smallFiles

Hope this helps :D

Important UPDATE: Starting in version 4.2, MongoDB removed smallfiles option.

More details at: https://docs.mongodb.com/manual/reference/program/mongos/#bin.mongos

So for latest available options, check the updated manual at: https://docs.mongodb.com/manual/tutorial/rotate-log-files/

Answer 2

Have a look at log rotation. mongo log rotation

It will allow you to automatically create multiple smaller log files, much like web server logs. Just delete the older ones as you go.

And as Chris points out, you just need a decent text processor that can accommodate 3GB files. The list is surprisingly short on those that properly memory map files, but notepad certainly isn't one of them as it just tries to load the whole file into RAM like many others. Alternatives are easy to find, but your mileage may vary: large gb text editors

Personally, I keep "tail -f" running on my Mongo log file and it's always running in an window I can refer to.

Answer 3

If you are in unix/linux , you can add

size 100M

in your

/etc/logrotate.d/mongod

no matter the interval the log will be rotated if it goes above 100M

Here is my config for the logrotate

/mongoTST/data/mongodb.log {
  daily
  rotate 5
  size 64M
  compress
  dateext
  missingok
  notifempty
  create 644 mongodb mongod
  sharedscripts
  postrotate
        /bin/kill -SIGUSR1 `cat /mongoTST/data/mongod.lock`
  endscript
}

Answer 4

As noted in cirrus' answer, MongoDB log rotation may be the best way to solve it.

To provide a bit more detail than is found in that answer, you could use the logrotate tool; this introduces 2 logs files and the small log lost window between copy and truncate. Or you could write a simple script to tell MongoDB to rotate the logs. mongologrotate.sh is a simple reference script. You could write a simple cron job or script to call it periodically like every 30 minutes.

Check the similar discussion on Stackoverflow.