1

Background:

  • Two empty shards with no explicit sharding scheme - shard0000 and shard0001.
  • mongorestore a single database, all of which defaults to shard0000
  • Add a shard key to one collection and let the balancer move 1171 chunks to shard0001, leaving 1171 on shard0000. (So far, so good.)
  • Manually move 2 chunks from shard0000 to shard0001, one of which happens to contain the highest values of the shard key. UPDATE: I had suspected this is where things got tricky, but I repeated the entire procedure described here without this manual move, and got the exact same results.
    • The result, according to sh.status({"verbose":true}) is both chunks simply having their assigned shard names changed from "shard0000" to "shard0001", including the row at the end of the output with { "key_field" : "highest_value" } -->> { "key_field" : { "$maxKey" : 1 } } on shard0001 Timestamp(1173, 0).
    • Totals are shard0000=1169 and shard0001=1173, as expected.
    • The only database listed has shard0000 as its primary.

Next steps:

  • Attempt to move all of the data back to shard0000 using db.runCommand({removeShard:"shard0001"})

The initial return value, as well as all subsequent reruns of the above command, documented to give the status of the removeShard operation, is

{
        "msg" : "draining ongoing",
        "state" : "ongoing",
        "remaining" : {
                "chunks" : NumberLong(1173),
                "dbs" : NumberLong(0)
        },
        "note" : "you need to drop or movePrimary these databases",
        "dbsToMove" : [ ],  <-- THE PROBLEM
        "ok" : 1
}
  • Considering that there is only one database, attempt to movePrimary it just in case. As expected, this returns { "ok" : 0, "errmsg" : "it is already the primary" }
  • sh.status() shows shard0001 as "draining" : true.
  • The "remaining" count does not change.
  • Moving back the two manually moved chunks to shard0000 does not affect the results.
  • There are no outstanding locks. db.locks.find({"state":2}) returns nothing.
  • The mongos log shows these two relevant operations, then nothing. (The shards' logs have no trace of this, with default verbosity.)
    • SHARDING [conn5] going to start draining shard: shard0001
    • SHARDING [conn5] about to log metadata event into changelog: { _id: ..., server: ..., clientAddr: ..., time: ..., what: "removeShard.start", ns: "", details: { shard: "shard0001" } }
  • No chunks are jumbo. db.chunks.find({shard: "shard0001","jumbo":true}) returns nothing.
  • db.runCommand({flushRouterConfig:1}) does not affect any of the above.

Question:

Is there really something I need to movePrimary here, despite the empty array for dbsToMove? If so, what is it and where did it come from? Have I made some more basic error in moving all of the data back to one shard?

Disclaimers:

  • There are several questions on SO about troubles with removeShard (example), most of which do not address specifically the case of dbsToMove: [ ].
  • The one that does (this one) has no answers, and I am hoping that by providing more details I can solicit one.
  • The documentation says "nor can you unshard a collection", which is essentially what I'm trying to do, but I wouldn't necessarily conclude from that that I can't drain a shard.
  • I found this bug ticket, but it doesn't seem directly related to this problem.

MongoDB 3.2.9

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Browse other questions tagged or ask your own question.