Questions tagged [mongodb]
All versions of MongoDB - a scalable, high-performance, open source, document-oriented database.
1,912
questions
75
votes
5answers
35k views
Which database could handle storage of billions/trillions of records?
We are looking at developing a tool to capture and analyze netflow data, of which we gather tremendous amounts of. Each day we capture about ~1.4 billion flow records which would look like this in ...
69
votes
4answers
58k views
Difference between Sharding And Replication on MongoDB
I am just confuse about the Sharding and Replication that how they works..According to Definition
Replication: A replica set in MongoDB is a group of mongod processes that maintain the same data ...
57
votes
4answers
148k views
How restore a specific database from backup using mongorestore command
I created a backup of all my databases using mongodump command. Now I want to restore a specific database using mongorestore command.
How is this possible, I use this command: --db option then ...
33
votes
3answers
35k views
check storage engine from shell
I'm upgrading to 3.0 and ran into some issues with the upgrade. Specifically, I got an error when trying to start up mongod via ssh, it tried to use the default dbpath instead of the one I specified ...
33
votes
1answer
50k views
Sample YAML Configuration Files for MongoDB?
The MongoDB configuration options documentation lists all the available options that can be specified, but does anyone have a set of fully formed example YAML formatted config files for MongoDB ...
32
votes
4answers
27k views
Difference between MongoDB's find and findone calls
I am working on a project and I am unsure if there is a difference between the way the find cursor works and the way the findOne cursor works. Is findOne just a wrapper for find().limit(1)? I was ...
28
votes
5answers
69k views
How can I enter Mongo as a superuser or reset users?
I'm was playing around with permissions and locked myself out of Mongo database. I'm pretty sure I did this by trying to explicitly add access to a database but instead I overwrote only allowing ...
28
votes
3answers
37k views
MongoDB using too much memory
We've been using MongoDB for several weeks now, the overall trend that we've seen has been that mongodb is using way too much memory (much more than the whole size of its dataset + indexes).
I've ...
26
votes
1answer
15k views
Mongodb incremental backups
I was given the task to set up incremental backups for MongoDB replicaset, as start point, of course, I googled it and could not find anything on MongoDB docs, I did find however this question on ...
25
votes
3answers
22k views
MongoDB MMAPv1 vs WiredTiger storage engines
In mongoDB3 appeared a new storage engine: WiredTiger. Yet, MMAPv1 is still the default choice in Mongo.
One might not be better than the other, it's often a matter of use case and choosing the ...
24
votes
2answers
12k views
Mongodump: Unrecognized field 'snapshot'
I am using mongodump from mongo tools to download a backup of my server. However as I run the command, the command returns "Unrecognized field 'snapshot'".
The full error is:
Failed: error reading ...
23
votes
1answer
20k views
What happens if there are too many inserts in MongoDB? How to ensure all data is stored?
I use MongoDB to store periodically measured values. Every ~100 ms a bunch of values is inserted as document. It works fine, but I'm worried about performance issues. (I use safe inserts, it seems as ...
23
votes
1answer
13k views
Using MongoDB and PostgreSQL together
My current project is essentially a run of the mill document management system.
That said, there are some wrinkles (surprise, surprise). While some of
the wrinkles are fairly specific to the project,...
20
votes
3answers
37k views
MongoDB: RAM requirements
Is it enough to have the entire index in memory/ram or does mongodb even try to allocate as much ram as possible to store even the data for fast reads?
I'd like to run mongodb + other applications ...
18
votes
2answers
1k views
CouchDB vs MongoDB [closed]
Evaluating document-oriented storage, what are the pros and cons of CouchDB vs MongoDB?
16
votes
5answers
4k views
Infrastructure for Highly Concurrent, High Write DB
My requirements are:
3000 Connections
70-85% Write vs Read
Currently, we are maxing out a High-CPU, Extra Large Instance at 700 connections. All 8 cores are maxed. We think it's the number of ...
16
votes
1answer
4k views
How do databases store index key values (on-disk) for variable length fields?
Context
This question pertains to the low-level implementation details of indexes in both SQL and NoSQL database systems. Actual structure of the index (B+ tree, hash, SSTable, etc.) is irrelevant as ...
14
votes
2answers
59k views
Mongo Create a user as admin for any database raise an error
I am trying to create a simple user with the rights permission to access to any database and can do any actions. When I trying to execute the createUser command I got this error:
db.createUser({ user:...
13
votes
2answers
18k views
How to backup large MongoDB database
What is the recommended way to back up large data sets in MongoDB? Let's say we have a data size in the order of 10TB - how would you back that up?
We're considering a hidden, possibly delayed, ...
13
votes
4answers
712 views
Good Resources on Operating/Administering MongoDB
What are the best resources for learning how to operate and administer MongoDB?
There are plenty of resources on developing against it - which actually creates a problem when you've got plenty of ...
13
votes
2answers
18k views
Mongo DB Replica set Stuck at RECOVERING state
We have created a replica set and now the problem is 2 members of replica set [3 members set] are in recovering mode from 48 hours. Initially the size of recovering nodes was increasing and now even ...
13
votes
2answers
4k views
Sudden Mongodb high connections/queues, db stops responding
The issue
We have a strange issue on our mongodb setup. Sometimes we get peaks of high connections and high queues and the mongodb process stops responding if we let the queues and connections ...
12
votes
3answers
84k views
Date range query for past 24 hour in Mongo shell
I am setting a cron job to collect results from MongoDB database profiler. I'd like to collect results within a 24 hrs period. I plan to run mongo command with javascript.
Question is, in Mongo shell,...
11
votes
2answers
6k views
MongoDB: co-locate the mongos process on application servers
I would like to ask a question about a best practice described in this document:
http://info.mongodb.com/rs/mongodb/images/MongoDB-Performance-Best-Practices.pdf
Use multiple query routers. Use ...
11
votes
4answers
15k views
Why is Mongo stuck in STARTUP2?
I have a Mongo replica set with a few secondaries. A box, which hosts a secondary instance, crashed and lost the database.
I started the secondary Mongo instance again and now it's stuck in STARTUP2 ...
11
votes
3answers
10k views
Upstart script in ubuntu for mongodb
I am new to linux and mongodb.
I've already installed mongodb on ubuntu but I want to run one more instance on different port.
As I understood from other questions and forums , there must be upstart ...
11
votes
3answers
13k views
Is it possible to install just the mongo Shell?
I have Docker installed and am running a MongoDB container for my local development on my Mac. The problem is that I can't connect to said DB easily from CLI. I have Robo 3T installed, but I would ...
11
votes
2answers
7k views
MongoDB Replica Set SECONDARY Stuck in `ROLLBACK` State
During a recent automated update of our mongodb PRIMARY, when the PRIMARY stepped down it permanently went into a ROLLBACK state.
After several hours in the ROLLBACK state, there was still no ...
11
votes
3answers
12k views
How should I store time series in mongodb
I need to create a database of time series, and perform the following tasks:
create new time series
update existing time series
query one or several time series at once (for instance all time series ...
11
votes
2answers
822 views
Is sharding effective for small collections?
It looks like database sharding is great if I have huge collections. What if I have lots of fairly sized collections ? Let's say that for 1 collection of 100 000 000 documents (not very big comments) ...
10
votes
9answers
47k views
MongoDB doesn't start after changing the data directory
I have installed a mongodb instance using yum.. Now everything works fine. I started the service using service mongod start. It works well. Then I changed the data directory and log path in the ...
10
votes
1answer
13k views
How do I (safely) kill long running operations in MongoDB?
Occasionally operations get out of control in MongoDB and might end up running for hundreds of seconds, and impact performance until they are killed or completed.
When that happens, I know I have ...
10
votes
2answers
7k views
MongoDB terminates when it runs out of memory
I have the following configuration:
a host machine that runs three docker containers:
MongoDB
Redis
A program using the previous two containers to store data
Both Redis and MongoDB are used to ...
10
votes
2answers
7k views
How do I determine why a MongoDB document is failing validation?
How do I determine why a MongoDB document insert is failing validation? All I get back is a writeError that says "Document failed validation", which isn't very helpful.
(This happens often, and I'd ...
10
votes
4answers
23k views
Error parsing YAML config file: yaml-cpp
I'm using the following command (Windows machine) with the MongoDB shell version: 3.0.7:
mongod --config "G:\NodeApps\mongod.cfg" --install
Contents of mongod.cfg file are given below:
systemLog:
...
10
votes
1answer
12k views
MongoDB performance vs. PostgreSQL with 5.5 million rows / documents
Can someone help me compare these queries and explain why the PostgreSQL query executes in just under 2000ms and the MongoDB aggregate query takes almost 9000ms and sometimes as high as 130K ms?
...
10
votes
1answer
22k views
Mongo cannot find my collection
My mongodb server has a database named villageContents
It has a collection named tablebusiness
If I run mongo I saw
MongoDB shell version: 2.0.7
connecting to: test
>
I wonder what "test" is. ...
9
votes
2answers
17k views
How to specify authentication database and target database separately on mongodb connection uri?
I am using this connection uri to connect to mongodb: mongodb://user:password@localhost/admin. It will use admin as bot authentication and target database. How can I make the uri to use admin as ...
9
votes
2answers
6k views
mongodb: replica-set with load balancer
Is there an option for load balancing with mongodb replica sets?
I am not asking for a HAProxy or similar, but for a built-in feature from mongodb.
Could not find tips for this in the docs.
9
votes
3answers
11k views
Backup with mongoexport or mongodump?
For production backup, MongoDB recommends mongodump instead of mongoexport for accuracy of data. However, I would need to "scrub" data off a MongoDB database before backing it up. I am not aware of ...
9
votes
1answer
14k views
mongodb config servers not in sync
I have setup with 2 shards, with 2 replica servers and 3 config servers, and 2 mongos. I have following problems:
1) mongo config servers out of sync:
Aug 14 09:46:48 server mongos.27017[10143]: Sun ...
9
votes
1answer
4k views
mongodb shard chunk migration 500GB takes 13 days - Is this slow or normal?
I have mongodb shard cluster, shard key is hashed. It has 2 shard replica sets. Each replica set has 2 machines.
I did an experiment by adding another 2 shard replica sets, and it starts to rebalance....
9
votes
2answers
7k views
Mongo Collection `Size` is *larger* than `storageSize`?
I recently compacted my collection using the command:
db.<collectionName>.runCommand( "compact" )
And now my collection size seems to be bigger than size on disk!
SECONDARY> db.<...
9
votes
1answer
7k views
MongoDB Not Using All Available RAM
I have something around 200 GB worth data stored in a mongo cluster. The physical memory on one of the instances which runs mongo is 8GB. Nothing else of any consequence runs on this instance. As near ...
8
votes
1answer
32k views
How to limit the connections per host in MongoDB?
I'm running a mongo server which accepts connections from other clients through java driver. The thing I have noticed is after a while some users open too many ports, and this prevents other users to ...
8
votes
2answers
749 views
Is there any way to efficiently perform the equivalent of DENSE_RANK in MongoDB?
SQL Server and Oracle both have DENSE_RANK functions. Is there a way to do something similar in MongoDB without having to resort to MapReduce? In other words, suppose you have a T-SQL select clause ...
8
votes
1answer
3k views
How to find MongoDB index statistics
Is there any way we can figure out what all indexes are cached into RAM ? I want to know page related stats for individual indexes (# of hits and miss )
8
votes
4answers
370 views
How to handle queries of 500M+ items
The structure of my data is the following:
date: <timestamp>
filter_a: <integer> -> range [0, 1000]
filter_b: <integer> -> range [0, 1000]
filter_c: <integer> -> ...
8
votes
1answer
4k views
How to index dynamic attributes in MongoDB
I have following kind of data (simplified a bit from my real case) in MongoDB:
{
"name":"some name",
"attrs":[
{"n":"subject","v":"Some subject"},
{"n":"description","v":"Some ...
7
votes
2answers
4k views
Mongodump affects app performance really bad
We have quite a big mongo instance (150 GB) with no sharding, and our regular backup (mongodump) has a very significant effect on the app performance. Worse than that, due to the heavy use of mongo by ...
