1

Currently, I have a existing mongo3.0 cluster(using WiredTiger storage engine and directoryPerDB is true) as below settings:

3 x shards(rs01, rs02, rs03)
2 x mongos
3 x mongo cfg

All the databases/collections in the 3 shards are disabled sharded. and for example, there are below databases in the 3 shards.

in rs01: db01, db03
in rs02: db02, db05
in rs03: db04

Now, I bought new hardware and want to setup a new cluster for better performance. According to the usage of all databases, I need to change the positions as below:

in new rs01: db02, db04
in new rs02: db01
in new rs03: db05, db03

All the databases data are several TB size(about 25TB in total.). I tried to use mongo dump/export is slow. It may take several weeks to transfer all databases to new cluster?

Although in the old cluster, all nodes config directoryPerDB: true, I tried to create an empty database named db01 in new rs02 and then insert a document in db01.co01, then copy the local directory /var/lib/mongodb/db01 in old rs01 to new rs02. But it doesn't work.

Is there any faster way to do this? Say, copying the local files to new cluster. If I can make the process fast, I'll announce the downtime of mongo service within 1 week and do it.

@Stennie mentioned about Migrate a Sharded Cluster to Different Hardware. However, this looks like node to node data path copy. In my case the position of all databases are redesigned for better performance, so I cannot do a node to node file copy.

  • do you use Open Source version? Your backup strategy uses mongodumpor binary backups? Is the commercial Ops Manager available? You do the migration to the same Linux flavour as before? – Ivanov Jan 11 '17 at 8:41
  • yes. I use mongo 3.0.12 on all cluster nodes with official ops manager to monitor health status. We don't have any backup strategy now :(. and original nodes and new nodes are all Ubuntu 16.04 OS. – CSJ Jan 12 '17 at 8:12
  • You definitely don't want to use mongodump to backup a large sharded cluster (or a sharded cluster, in general) as it will be slow to dump (all data must be read by mongod) and require quite a few manual steps to restore into a new cluster. Are your servers bare metal or VMs? Are they physically located in the same data centre? Typically hardware upgrades can be done in a rolling fashion to avoid downtime as per Migrate a Sharded Cluster to Different Hardware. – Stennie Jan 12 '17 at 15:53
  • current mongo cluster nodes are running in VMs, and we bought a new batch of physical machines for new cluster which are all in out datacenter/LAN. I read the Migrate a Sharded Cluster to Different Hardware if I don't want to stop current cluster, and change the databases position in new cluster, seems not mentions about this topic – CSJ Jan 17 '17 at 8:47
  • if I add the new cluster replica set into current cluster as (rs04, rs05, rs06). and use db.runCommand( { movePrimary: db01, to: rs05 } ), is it work? and users can still read/write while the command running ? – CSJ Jan 17 '17 at 8:49

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Browse other questions tagged or ask your own question.