0

I have a running MongoDb replicaset consisting of 2 replicas and 1 master nodes. Recently I noticed that my master gone and automatic failover took action and changed master, however I wasn't aware of this. Furthermore, consequently backups are missing for a few days as the mongo service was doen on master and backup was set up only on master.

Obviously this isn't a best practice as I have nor alerts neither proper backups.

I would like to ask couple of things regarding this.

  1. Is it fine to set up backups script on all 3 nodes with plain crontab and mongorestore?

  2. How usually alerting is done in this kind of situations when I have to know status of MongoDB (or any other service generally) based on some events like termination or high load?

FYI I leverage MongoDB on AWS EC2 Ubuntu instances.

Thanks in advance!

1

Yes, you can run backup on all nodes (but rather mongodump or monogexport). It is just a matter of available disk space.

You can develop your own scripts and check whatever you like, e.g. systemctl status mongod, rs.status(), db.hello(), etc.

Or you use the Mongo Ops Manager

And then you have standard monitoring tools, one of the most common known is Nagios

0

It's always recommended to run backup only on the SECONDARY nodes. Running backup on primary node will increase the disk IO since we already have heavy writes traffic from the application to the PRIMARY node. You can write your own shell/python scripts to configure based on the below logic:

  1. Identify whether node in the replica set is SECONDARY or not. If the below returns false then it's secondary and you can proceed to next step for backup.

db.hello().isWritablePrimary

  1. Run the mongodump backup commands,compress the dump file and save it on some location

mongodump --host hostname --db dbname --archive=/db_backup/dbname.gz --gzip

To alert you can try few open source plugins for basic DB health checks: Mongo Monitor, Check MongoDB.

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.