4,046 reputation
422
bio website comerford.cc
location Ireland
age 35
visits member for 2 years, 11 months
seen 14 hours ago

I currently work for MongoDB, Inc. and have a background in operations, networking and system administration. You can find me in all the usual places, but probably the most useful and relevant are listed below:

Personal Site/Blog
LinkedIn
Twitter
Stack Overflow Careers


Jan
27
answered Setup MongoDB replica servers with different power?
Jan
20
answered Is there any way to change the size of preallocate data file in mongodb?
Jan
10
comment Mongodb replication behind a gateway
Your diagram does not imply a gateway, it implies a proxy, or at the very least NAT traversal and port changes. If the port differences are transparent then it will work. If not......the key point is: can server1/2 open a TCP connection to s3.some.org on port 27017 and vice versa? If the answer is no, then this will not work without extra configuration. You can make different IP addressing work with clever use of hosts files or DNS but if you are going to have different ports depending on location you will need to employ some sort of tunneling to map from one to the other for consistency
Jan
8
comment mongoexport not exporting all possible data
I can try to reproduce, but would need to know what indexes you have on that collection - can you add that information to the question? Can you also comment on the lower/upper bound for the integer in the id field?
Jan
8
comment Mongo db sharded “the collection's metadata lock is taken” issue coming very frequently
to troubleshoot this fully and get to a root cause I would need to see logs from the mongos processes, the mongod processes, a copy of the config database and would need to get an idea of your traffic pattern for inserts, how article IDs are generated. Basically this is a full blown support or consulting case and I don't think it will be possible to solve it on stack exchange. The procedure I have outlined above will work, but you will need to apply it judiciously to your particular cluster (perhaps programmatically). The only other option is to revert the settings and wait for the balancer
Jan
8
answered MongoDB - real disk usage is LOWER than dbStats.storageFile
Jan
8
comment Mongo db sharded “the collection's metadata lock is taken” issue coming very frequently
The too big message is expected if you lower your chunk size - you will have a large portion of chunks that exceed the new size as a result and will need to be split. You can either put the chunk size back up (average doc size x 250000 should cover most based on your previous comments) or look to address the issues manually. I can't say for sure without in depth troubleshooting that really isn't possible here but I suspect the root issue here is shard key related
Jan
7
comment Sample YAML Configuration Files for MongoDB?
Just in case anyone wants a setParameter example, see this answer: dba.stackexchange.com/a/87653/6441
Jan
7
comment Mongo db sharded “the collection's metadata lock is taken” issue coming very frequently
There is no point in setting a maximum chunk size of anything over ~200MB if your documents in that chunk are less than 1024 bytes in size. If they are, then the chunk will always split at the 250k document mark anyway and the size setting is basically meaningless. There are very few reasons to alter the setting from the default of 64MB so I am not sure why you have done so here. In any case, it will mean longer chunk migrations and may be contributing to your issues, I would recommend reverting to the default size
Jan
7
revised Mongo db sharded “the collection's metadata lock is taken” issue coming very frequently
Added explanation, initial step to try
Jan
6
answered Mongo db sharded “the collection's metadata lock is taken” issue coming very frequently
Jan
6
comment Mongo db sharded “the collection's metadata lock is taken” issue coming very frequently
The balancer is actually not responsible for splits (it moves chunks after they are split to address imbalances), the mongos processes are what attempt to split chunks based on the amount of writes (inserts/updates) they see to a given chunk. Let me put together the query you need to run to get the specific locks for this collection, in the mean time I will add an interim answer with some steps to try to address the immediate issue
Jan
6
comment Mongo db sharded “the collection's metadata lock is taken” issue coming very frequently
Please post the output of the following to the question (from a mongos): use config; db.locks.find(); - the warning you are seeing basically implies that the meta data for the collection in question is locked, so we need to see where that lock is and what is holding it. Occasionally locks can go stale, but there is logic in MongoDB to force them eventually (it can take time), at which point the collection can split and then be balanced (currently there is only one chunk because splits are failing and hence no balancing is possible)
Jan
6
answered How to add setParameter in mongod config?
Jan
6
comment Why mongod said “note: noprealloc may hurt performance in many applications”?
Question for you - what OS and filesystem were you using? I just tried this on Linux with 2.6.6 on ext4 and was unable to reproduce
Jan
6
comment Why mongod said “note: noprealloc may hurt performance in many applications”?
I thought we had caught all of these (setting a default causing it to flip). Let me see if I can reproduce and I'll file a ticket if so
Dec
29
comment How to use fsyncLock() safely in backing up a single mongod
The on disk format has been the same for as many versions as I can remember, there will be a new format with the WiredTiger storage engine in 2.8 but until then, if you copy the data files (data and namespace files) for a database between different instances they will be usable, yes.
Dec
25
answered How to use fsyncLock() safely in backing up a single mongod
Dec
25
answered mongoexport not exporting all possible data
Dec
23
answered Mongodb: Possible to enlarge a capped collection?