0

Running a mongodb 2.6 cluster on localhost with four shards and four replication sets.

Using a for loop with 100 iterations that pre-splits chunks among the existing shards. I am getting the error The collections metadata lock is taken after a couple of minutes

mongos> for( var i=0; i < 100; i++) { sh.splitAt("test_cluster.presplit_demo", { x:1000000 * i / 100 } ); }
{
    "cause" : {
        "who" : {
            "_id" : "test_cluster.presplit_demo",
            "state" : 2,
            "who" : "localhost.localdomain:27000:1402761131:1123537020:conn19:1622303703",
            "process" : "localhost.localdomain:27000:1402761131:1123537020",
            mongos&gt; for( var i=0; i &lt; 100; i++) { sh.splitAt("test_cluster.presplit_demo", { x:1000000 * i / 100 } ); }
{
    "cause" : {
        "who" : {
            "_id" : "test_cluster.presplit_demo",
            "state" : 2,
            "who" : "localhost.localdomain:27000:1402761131:1123537020:conn19:1622303703",
            "process" : "localhost.localdomain:27000:1402761131:1123537020",
            "when" : ISODate("2014-06-14T15:56:16.439Z"),
            "why" : "migrate-{ x: 360000.0 }",
            "ts" : ObjectId("539c70a07b296ac1c721eca4")
        },
        "ok" : 0,
        "errmsg" : "the collection's metadata lock is taken"
    },
    "ok" : 0,
    "errmsg" : "split failed"
}

I see the lock in config: db.locks.find(), removed the lock but it just comes back. If I reduce the number of iterations to 50, the for loop completes successfully. There is more than enough disk space.

Question

Anyone else run into this issue with pre-splitting and what did you do to resolve it?

1

Turn off the balancer first, you are likely seeing it trying to balance the collections while you are attempting to split it. You can do this with the sh.stopBalancer() command. If you take a look at this answer I go into the necessary steps for pre-splitting in more detail.

  • Thanks Adam, I read through the other answer you provided. A shame that turning off the balancer isn't mentionned in MongoDB 102. – Craig Efrein Jun 16 '14 at 5:47
  • It's not 100% necessary, but it's a good idea - really what should be done is that any script that is looking to change meta data in any way should check for locking and back off until it is allowed to make changes. But, it's far less complicated for simple tasks to just turn off your competitor. I did mention turning off the balancer in M202, though I may not be explicit about it as a requirement - not sure about it being mentioned in M102 – Adam C Jun 16 '14 at 8:47

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.