MongoDB : Document-Oriented NoSQL Database
28
votes
5answers
2k views
Which database could handle storage of billions/trillions of records?
We are looking at developing a tool to capture and analyze netflow data, of which we gather tremendous amounts of. Each day we capture about ~1.4 billion flow records which would look like this in ...
13
votes
2answers
842 views
CouchDB vs MongoDB [closed]
Evaluating document-oriented storage, what are the pros and cons of CouchDB vs MongoDB?
10
votes
2answers
319 views
Is sharding effective for small collections?
It looks like database sharding is great if I have huge collections. What if I have lots of fairly sized collections ? Let's say that for 1 collection of 100 000 000 documents (not very big comments) ...
10
votes
1answer
878 views
How do databases store index key values (on-disk) for variable length fields?
Context
This question pertains to the low-level implementation details of indexes in both SQL and NoSQL database systems. Actual structure of the index (B+ tree, hash, SSTable, etc.) is irrelevant as ...
9
votes
3answers
410 views
Good Resources on Operating/Administering MongoDB
What are the best resources for learning how to operate and administer MongoDB?
There are plenty of resources on developing against it - which actually creates a problem when you've got plenty of ...
9
votes
5answers
686 views
Infrastructure for Highly Concurrent, High Write DB
My requirements are:
3000 Connections
70-85% Write vs Read
Currently, we are maxing out a High-CPU, Extra Large Instance at 700 connections. All 8 cores are maxed. We think it's the number of ...
8
votes
2answers
2k views
Difference between MongoDB's find and findone calls
I am working on a project and I am unsure if there is a difference between the way the find cursor works and the way the findOne cursor works. Is findOne just a wrapper for find().limit(1)? I was ...
8
votes
2answers
822 views
MongoDB Replica Set SECONDARY Stuck in `ROLLBACK` State
During a recent automated update of our mongodb PRIMARY, when the PRIMARY stepped down it permanently went into a ROLLBACK state.
After several hours in the ROLLBACK state, there was still no ...
6
votes
2answers
200 views
Is there any way to efficiently perform the equivalent of DENSE_RANK in MongoDB?
SQL Server and Oracle both have DENSE_RANK functions. Is there a way to do something similar in MongoDB without having to resort to MapReduce? In other words, suppose you have a T-SQL select clause ...
6
votes
2answers
1k views
mongodb: replica-set with load balancer
Is there an option for load balancing with mongodb replica sets?
I am not asking for a HAProxy or similar, but for a built-in feature from mongodb.
Could not find tips for this in the docs.
6
votes
1answer
532 views
Mongo Map-Reduce or Sharding?
So I asked this question:
Setting up Mongo with clustering
But I didn't know enough until I read the answers and did more research. Is it better to use Map-reduce or sharding for fast-paced ...
6
votes
1answer
56 views
Overview of how MongoDB uses its various threads
On one instance I have MongoDB using ~85 threads. In lieu of having time to investigate directly, I am curious:
What are the threads used for? Do they fall into a few high-level usage categories?
...
5
votes
3answers
1k views
Upstart script in ubuntu for mongodb
I am new to linux and mongodb.
I've already installed mongodb on ubuntu but I want to run one more instance on different port.
As I understood from other questions and forums , there must be upstart ...
5
votes
2answers
2k views
How should I store time series in mongodb
I need to create a database of time series, and perform the following tasks:
create new time series
update existing time series
query one or several time series at once (for instance all time series ...
5
votes
3answers
754 views
Setting up Mongo with clustering
I'm going to start using Mongo as a database but I will not have it clustered at first. Can I use Mongo on a single machine, then later come back, cluster it and capitalize on the sharding? Or can I ...
5
votes
1answer
80 views
Are shards distributed randomly across machines in MongoDB or are they in order?
Are shards distributed randomly across machines in MongoDB or are they in order?
For example, if I had a MongoDB cluster that was sharded across two machines and my shard key was a person's name, ...
5
votes
1answer
212 views
MongoDB deployment best practices
I had a few questions around deployment best practices and how folks are using mongodb today. Feel free to add any other comments you can think off.
No of databases per mongod instance. Is the ...
5
votes
1answer
726 views
What factors contribute to lock-percentage with MongoDB
We're attempting to better optimize how we're using our MongoDB instance. We seem to be routinely getting high lock-percentages, and are looking to help minimize that. Here is some mongostat output:
...
5
votes
1answer
174 views
How to store and query products the price of which depends on a complex set of conditions?
I haven't decided on a database. I have expirience with MySQl and I'm intrigued by MongoDB.
Let the products be packages of cable tv providers:
{
name: "Virgin TV",
price: 20
},
{
name: ...
5
votes
1answer
593 views
Using MongoDB and PostgreSQL together
My current project is essentially a run of the mill document management system.
That said, there are some wrinkles (surprise, surprise). While some of
the wrinkles are fairly specific to the ...
5
votes
1answer
447 views
In MongoDB, is having lots of small collections better than having one huge collection?
Huge collections have huge indexes that may not fit into RAM. Is that problem entirely solved with sharding ?
If I have lots of small collections with small indexes, what are the options for storing ...
4
votes
2answers
914 views
MongoDB: RAM requirements
Is it enough to have the entire index in memory/ram or does mongodb even try to allocate as much ram as possible to store even the data for fast reads?
I'd like to run mongodb + other applications ...
4
votes
1answer
235 views
Would appreciate some suggestions on how I should model this data
I am setting up a website that will have users. Users can search for songs and (if signed in) save them to a playlist or their library of songs.
I want it to be like iTunes in that all the songs on ...
4
votes
1answer
140 views
what is mongodb telling me when it logs an update?
After several minutes of a large program that pushes a lot of data into mongo, my log starts to show a message for (I guess) each update. This seems awfully noisy, is there some special reason it ...
4
votes
3answers
2k views
How can I time-stamp data in MongoDB?
I'm responsible for writing an app to juggle a huge amount of client and business intelligence information. This information involves records for various businesses and metrics like revenue, their web ...
4
votes
1answer
317 views
MongoDB optimization on high data
I have a mongoDB collection with 42M registers with 1M new registers every day.
We found a low performance on mongoDB querying.
showing in mongostat the fields: qr|qw ar|aw with values of 300-500.
...
4
votes
1answer
463 views
Mongo Collection `Size` is *larger* than `storageSize`?
I recently compacted my collection using the command:
db.<collectionName>.runCommand( "compact" )
And now my collection size seems to be bigger than size on disk!
SECONDARY> ...
4
votes
1answer
177 views
MongoDB working set size on capped collection larger than capped collection size?
On my database server right now I have 16GB of ram and an 8GB capped collection with about 4GB of indexes. So I may understand this wrong but shouldn't this only ever use around 12GB of my RAM or ...
4
votes
1answer
392 views
Design the big data with mongo db
I'm working now on the implementation of the following feature:
Say I have a collection in mongodb with millions of entries (very big say 100M), each entry is of type
{"uniqId", "properties"}
for ...
4
votes
1answer
373 views
How to index dynamic attributes in MongoDB
I have following kind of data (simplified a bit from my real case) in MongoDB:
{
"name":"some name",
"attrs":[
{"n":"subject","v":"Some subject"},
{"n":"description","v":"Some ...
4
votes
1answer
41 views
MongoDB problems recovering a member of the replica set
I have a sharded database with 2 replica sets (RS1 and RS2) each one of the RSs with 2 servers. I had a problem yesterday with one member of the RS2, the mongod instance crashed throwing an error. ...
4
votes
0answers
104 views
Management tool for managing MongoDB sharded cluster
Is there a database management software for MongoDB that makes it easy to create a cluster, add shards, backup shards and all the complicated stuff. There are some many MongoDB clients but they are ...
3
votes
2answers
151 views
Does MongoDB auto compacting
I copy a collection to another collection. Data is 14GB. I deleted the copied collection. Yet the size of the directory is still 14 GB.
I chose repair but it seems to take forever. Actually inserting ...
3
votes
2answers
265 views
MongoDB: Replica Set - master vs. slave
We think about moving from a master-slaves to mongodb with replica-set.
Now I wonder if mongodb replicas are also set up like read-only slaves or if each mongos is able to write and sync the others ...
3
votes
1answer
67 views
Sharded key-value store using MongoDB
Would like to set up a key-value store that is sharded across multiple machines.
We are currently using MongoDB, is there a reason why we shouldn't use MongoDB for this purpose?
We also use Redis, ...
3
votes
2answers
4k views
Error: couldn't connect to server 127.0.0.1:27017 src/mongo/shell/mongo.js:91 when changing mongodb data directory
I already change the owner of the directory I want to use
root@ip-10-138-30-205:/media/newdrive# ls -l ...
3
votes
1answer
1k views
Mongodb - Returning the points nearest to a location, with distance
What is the MongoDB's equivalent to this query:
SELECT 111151.29341326 * SQRT( pow(-6.186753-`Latitude`, 2)
+ pow(106.772835-`Longitude`, 2)
...
3
votes
1answer
431 views
Mongo cannot find my collection
My mongodb server has a database named villageContents
It has a collection named tablebusiness
If I run mongo I saw
MongoDB shell version: 2.0.7
connecting to: test
>
I wonder what "test" is. ...
3
votes
1answer
91 views
Does mongodb automatically move small non-partitoned databases to spread read load?
I have a few hundred small databases, each between 0.2GB and 4.0GB in size. These are sitting on a sharded mongodb environment with 10 or so shards.
At any one time, only a very small subset of these ...
3
votes
2answers
393 views
Import and Export database in Mongo just like MySQL
I am newbie to the Mongo.Before this I have worked only with MySQL.In MySQL we can import and export database easily.I want to know is there any option available in Mongo to import and export database ...
3
votes
1answer
260 views
MongoDB Not Using All Available RAM
I have something around 200 GB worth data stored in a mongo cluster. The physical memory on one of the instances which runs mongo is 8GB. Nothing else of any consequence runs on this instance. As near ...
3
votes
1answer
514 views
MongoDB - handling embedded documents and relations
I am researching MongoDB for a web app that I am bulding. Coming from a MySQL background, the concept of embedded documents is not so easy to fully understand.
Let's say I have a document called ...
3
votes
1answer
87 views
MongoDB: Should I combine fields for performance in limited circumstances
I have a really simple MongoDB collection:
{
"user": 1,
"domain": "example.com",
"apiAccess": true
}
There's an index on the "user" and "domain" fields. I always perform the same type of ...
3
votes
2answers
168 views
What are the best Database Analytics Companies for analyzing several 1000s or Mio. of products,sum them together and analyze them for statistics?
We will get several thousands products in our database. The problem is that one product can have several different names e.g. "iPad 16GByte UMTS schwarz" or "iPad 16GB 3G black" and so on. But they ...
3
votes
1answer
194 views
Ideal environment and cost for running a medium size MongoDb server
I know some people will say this question does not belong here but i didnt find a better place on the internet to ask this. Please migrate to an appropriate place if you think it should be.
I am using ...
3
votes
1answer
306 views
MongoDB: queue vs. realtime write/update
On a large mongoDB I have 2 different use-cases for reading and 2 different types for write/update.
read 1: find by index -> no question on this. this works like a charm - as long as the index fits ...
3
votes
1answer
271 views
Could a large log file be the reason cron job keeps dying?
The log is 15 Gigabytes. I guess I have to back it up and delete. I have enough hard disc space, but did not realize log was so large. And overnight cron jobs suddenly started to die in the middle.
3
votes
1answer
146 views
How to check the pre-allocated database file of MongoDB is Used or not?
We have been using MongoDB for a few weeks. One thing we could not understood is Pre-allocated file. Is there any way to check whether the pre-allocated file is currently used by MongoDB or not?
We ...
3
votes
1answer
28 views
Primary replica set server goes secondary after secondary fails
I have a 2 servers replica set that, after the secondary fails the primary goes into secondary mode while the secondary is in STARTUP2 (recovering). The problem with this is that I can't use the ...
2
votes
1answer
1k views
Error “cannot connect to server” installing MongoDB on Ubuntu
I'm trying to install MongoDB on Ubuntu 10.04.
I followed these installation instructions. I got the following error:
Error: couldn't connect to server 127.0.0.1:27017 src/mongo/shell/mongo.js:91
...
