Sharding is the practice of logically dividing or partitioning data, usually using a sprecific key (referred to as a shard key), and then placing that data on separate hosts (subsequently known as shards). This is generally done to scale horizontally (more hosts) as opposed to vertically (more ...

learn more… | top users | synonyms

0
votes
0answers
14 views

Expanding MariaDB cluster (sharding)

I have an expanding MariaDB 3 node cluster and need to expand the available storage. Each node has (to make the maths slightly easier) 120Gb of database storage. As this is replicated, the total DB ...
0
votes
1answer
13 views

Adding more space + bandwidth to a MongoDB cluster

I've read a lot about sharding and have gone through cluster setup process as well. What I can't get my head around is the actual database capacity after its sharded. So say for instance I have one ...
3
votes
1answer
38 views

How to make most of aws?

I'm new to aws (just one day old). I'm impressed with aws free tier and want to utilize to its best. As of now I'm done with installing mongodb on aws micro instances and mongod EC2 instances are ...
0
votes
0answers
14 views

mongodb balancer exception

We are using mongodb having 2 shards.But we realized that chunks not moving between shards.When I look at logs of servers running mongos we are seeing the message: SHARDING [Balancer] balancer move ...
0
votes
1answer
28 views

Must data be evenly distributed among Mongo shards?

According to this article: https://www.mongodb.com/blog/post/on-selecting-a-shard-key-for-mongodb. Assuming I have my schema as: { time_posted: ..., userid: ..., content: ... } The ...
1
vote
0answers
25 views

How does defining a MongoDB shard key influence the connectivity between my mongo(s/d)s?

I am trying to test mongoDB sharding, but when following the basic steps in common tutorials, I am facing a CONNECT_ERROR issue that really doesn't make sense to me. I use four machines for this test ...
6
votes
1answer
14 views

Mongos not returning existing data in shards

So we have a configuration of 2 shard (each is a replica set) and one sharded collection. There are around 10 million records in each shard. We are using mongo 2.6.4 version. We have sharded with ...
1
vote
0answers
70 views

Securing High-Availability on Partitioned Database

I have a 90GB SQL 2012 database I am sharding over 3 database and I plan on growing as more clients are coming. A new customer came to us with a demand to know on we secure high-availability in our ...
0
votes
1answer
30 views

How set balancing as main task for sharder MongoDB cluster?

I have a big collection (over 400 millions records) which I want to divided between 4 shards. I created infrastructure (4 shards, config server, mongos) and all work well (balancer is working and data ...
0
votes
0answers
29 views

mongodb: fail for removing shard from shardings

When I try to remove a shard from my mongodb sharding, I got the following error: Sun Sep 6 11:30:09 [Balancer] going to move { _id: ...
0
votes
0answers
15 views

mongodb sharding by id, based on the first 8 characters

we have a database full of grams,and they're organised in a tree fashion, with parent references, like so: _id: gram:a;p1:as;p2;astr;p3;astrot;p4:astrotur;word:astroturf we would like to chunk the ...
2
votes
1answer
54 views

How to deploy MongoDB config servers to achieve maximum availability during network partitions?

What's the recommended way to deploy config servers? Assume, we have two shards, each one is a 3-nodes replica set. If I deploy all config servers together with one of these shards, a single network ...
0
votes
1answer
44 views

No Data Replication in Progress mongodb

Setup Information One ConfigServer mongod.exe --configsvr --dbpath C:\MongoDBCluster\ConfigSrv --Mongos Server-- mongos.exe --configdb 10.133.1.130) 2 Shard Servers mongod.exe --shardsvr ...
1
vote
1answer
123 views

MongoDB: movechunk failed after adding a new share to cluster

I have a MongoDB cluster version 2.6.8, which already has two shares: share1 and share2. When I add a new share3 to the cluster, data chunk can't migrate to share3. This is from the mongos log: ...
2
votes
2answers
114 views

Sizing a MongoDB shard?

I am in the process of planning a MongoDB deployment, this will be my first to production and I have a question regarding the disk size of each shard in a cluster. This cluster itself will be for ...
1
vote
0answers
38 views

Database sharding, denormalization and synchronization

In sharding, it is advisable to follow a share nothing, shard everything approach. Essentially, this translates to: data that are to be used together are to be stored together (i.e. in a single ...
2
votes
1answer
26 views

Sharding Mongodb while limiting the number of documents to be migrated

I have little experience working with databases, but at my work i was tasked to deploy a sharded Mongodb cluster and the constrain on the shard is: database with multiple colections, 7 collections ...
0
votes
2answers
40 views

Wrong sharding on mongoDB 3.0

I have a mongos with 2 shard servers (and 3 config servers) sh.status() --- Sharding Status --- sharding version: { "_id" : 1, "minCompatibleVersion" : 5, "currentVersion" : ...
2
votes
1answer
73 views

MongoDB sharding

I have a questions about mongodb sharding. I have 7 VM with 1 master mongos and 3 config server en 3 shard servers. Mongo Main - 192.168.212.173 poort 27020 Mongo Shard1 - ...
0
votes
0answers
116 views

Mongodb does not contain shardkey for pattern

I've created a database with 3 sharded clusters using the wiredTiger engine and I've enable the storage engine option for replicas. Now after enabling sharding for collection, importing a csv to mongo ...
0
votes
1answer
68 views

updating single row in MYSQL

We have a situation where we need to update on a single row several thousands time a second and this requirement lasts continuously for 5 hours every day due to the nature of the business. Design ...
0
votes
0answers
26 views

MySQL 5.6 or 5.7 sharding geospatial data

I have ran into a problem where I need to start to scale horizontally. With my database I am performing only nearest point searches with my data. So from a lat/long inoput I find all the nearest ...
0
votes
0answers
56 views

mysql sharding techniques

I have developed one website for social network and I have used mysql for data storage. Now a days, many tables become a very huge (2-3 tables has 30 million records & 3-4 table has 300k records) ...
6
votes
1answer
95 views

Remove a shard in MongoDB

I have a MongoDB cluster, I accidentally hosed a shard, I don't have a backup. It's pretty easy to recover the data, but after issuing the removeShard command, it says it's draining. However, the ...
0
votes
1answer
93 views

Why sharding does not show any improvement?

Here is what I did to test sharding performance. I have 4 servers, I ran config server and router on a server. 3 others are meant to be shards. First, I only added one shard to router, then I ran a ...
1
vote
1answer
134 views

MongoDB : How to estimate max number of connections supported?

We have a sharded cluster with 2 replica sets. Each replica set is a 3 member cluster with one primary and 2 secondaries. Each instance is a r3.large (aws-ec2) instance with 15G ram. mongos's are in ...
2
votes
2answers
1k views

Mongos version 3.0.1 Split Chunk Error with Sharding

I have 3 config server and two shards. Each shard has a primary, secondary and an arbiter. This is running with the following mongod settings: storageEngine=wiredTiger ...
0
votes
0answers
30 views

MongoDB sharding one collection on single server (2 drives)

I have one single server with two drives (SSD 256GB, HDD 1 TB). What I want to do is to store the newest documents in collection on SSD. Older documents can be stored in HDD drive. Is it possible in ...
0
votes
1answer
96 views

MongoDB Hashed Shard Keys Selection

I have a DB which holds very simple documents called "Item": {_id:<ObjectId>, playerId:int, isRead:bool, content:string} containing about 80M documents. Each Item belongs to a certain player, ...
1
vote
1answer
47 views

How to shard a database like that of SO?

I know a bit about sharding but I cannot wrap my head around that fact that say eventually SO cannot scale vertically then how would they shard their SQL DB? Even if we assume SO will not reach that ...
0
votes
1answer
67 views

Is a update-only-once-row table worth sharding?

I have a big MySQL 5.6 Inno DB table that contains user's request. A table row updates only once, shortly-after the initial insert! (The initial insert is to get the unique auto-incrementing ...
0
votes
1answer
394 views

Can not drop db in sharded environment

I am trying to delete some old dbs in my sharded set up. Step i am taking are : Stop Balancer. Drop Db Start balancer But in some of the DBs i am getting these errors. "errmsg" : "exception: ...
0
votes
0answers
129 views

Mongo Chunk 250000 document limit is creating problem in chunk migration

We can set chunk size limit for a DB. But chunk has a limit of 250000 documents per chunk. My chunk size is 250 MB which varies the number of document in each collections. So normally some of my ...
1
vote
1answer
649 views

Mongo db sharded “the collection's metadata lock is taken” issue coming very frequently

I am seeing this warning in mongoS log consistently for one collection. [conn96] warning: splitChunk failed - cmd: { splitChunk: "ibeat20150105.dgrpCount", keyPattern: { articleId: 1, host: 1 }, min: ...
5
votes
1answer
317 views

Chunk Size is shown as 1 KB in mongo db logs even it is set to 300 MB

I am running a mongo cluster. Chunk size set is 300 MB but for today morning it is showing me in logs taht chunk size is 1024 Byte. I checked in current op there also it is showing chunks of 1024 ...
1
vote
2answers
218 views

Help - Error : Could not connect to master of shard replica set

I have a shard, using mongo v2.4.11. It has to mongos, 3 config, and 8 replica set (2 replica for each set, means 16 node for shard) It is keep having exception as per below. Caused by: ...
0
votes
1answer
503 views

I can not copy database MongoDB

I created a mongo cluster in ubuntu and run the copy in the sharding with a small database, everything works fine. The problem is that when I try to copy a database 300gb, is not getting done. The ...
0
votes
1answer
59 views

MongoDB Sharding : What should be the Sharding key if we want to find records with more than one field

I am having collection ("links") with listed fields: collection-name: "links" 1) url 2) title 3) score 4) description 5) ratings (embedded_many) and having index on "url", "title", "score" because ...
2
votes
1answer
348 views

MongoDB Stuck with Jumbo chunks while exhausting a shard

I'm running a shared MongoDB 2.4 cluster. I'm currently in a process of exhausting a shard and I'm down to the last 15 chunks needs to be migrated out of 9660. My issue is that these chunks are too ...
1
vote
1answer
171 views

Choosing shard key with compound index field

I have a compound key with field articleId_1_host_1_timeStamp_1_isIndia_1_location_1. Cardinality of fields is articleId has 20000 different value host has 17 value timestamp is increasing field ...
1
vote
2answers
93 views

Query to Secondary on Sharded Cluster

How to specify query from mongos shell which is inteded to query data from secondary? The goal is not to bother primary load on write and use data in secondary shards instead.
0
votes
1answer
86 views

MongoDB Tag-Aware Sharding Workflow

I wonder the work flow of sharding cluster in Tag-Aware Sharding. --- Sharding Status --- sharding version: { "_id" : 1, "version" : 3, "minCompatibleVersion" : 3, "currentVersion" ...
0
votes
1answer
278 views

MongoDB - Change primary shard only for a certain database

I have a sharded MongoDB cluster running 3 different Databases. Currently all 3 databases has the same shard configured as their primary shard (shard000) Is it possible to change the primary shard for ...
1
vote
1answer
160 views

Repair database on Replica in sharded set up

I am using mongo sharding in my DB. The scenario I am facing is that I have deleted some of my collections in my DB. But as to reclaim the space I need to run repair option which i can not run using ...
1
vote
1answer
609 views

How to update a field in sharded environment in mongodb using C# driver

I have successfully done loading in sharded environment. It is working fine. But when I do update in sharded environment, I am getting the error as : WriteConcern detected an error 'update { q: { ...
0
votes
0answers
20 views

Mongodb sharding balancing not working properly, says config servers differ [duplicate]

Iam using a mongodb sharding configuration in which there are 3 config servers, 2 query routers and 2 shards. One of my shard is having 32 gb data and other is having only 6 gb data. Balancing is not ...
1
vote
0answers
123 views

Problems in sharding in mongodb

I am currently working on sharding. I have certain issues related to sharding : 1.If I load data again to a collection which is already sharded, the data stays only in one shard and not migrating to ...
0
votes
1answer
79 views

Migration of chunks not taking place in sharding

I am trying sharding in MongoDB. I am working with 1 config server 1 Query router 2 shards I tried hash based sharding. The data is getting partitioned but sharding is not successful. In one ...
4
votes
1answer
974 views

mongodb shard chunk migration 500GB takes 13 days - Is this slow or normal?

I have mongodb shard cluster, shard key is hashed. It has 2 shard replica sets. Each replica set has 2 machines. I did an experiment by adding another 2 shard replica sets, and it starts to ...
1
vote
1answer
387 views

Is it correct to use IP addresses (instead of hostnames) while configuring Mongo DB cluster? Does it cause any issues?

I have been using Mongo DB Cluster for some data analysis (Mongo DB version 2.4.5). My configuration is as follows: 1) Node1 - mongoconfig server, mongod shard 2) Node2 - mongoconfig server, mongod ...