Sharding is the practice of logically dividing or partitioning data, usually using a sprecific key (referred to as a shard key), and then placing that data on separate hosts (subsequently known as shards). This is generally done to scale horizontally (more hosts) as opposed to vertically (more ...

learn more… | top users | synonyms

0
votes
2answers
29 views

Wrong sharding on mongoDB 3.0

I have a mongos with 2 shard servers (and 3 config servers) sh.status() --- Sharding Status --- sharding version: { "_id" : 1, "minCompatibleVersion" : 5, "currentVersion" : ...
0
votes
1answer
33 views

MongoDB sharding

I have a questions about mongodb sharding. I have 7 VM with 1 master mongos and 3 config server en 3 shard servers. Mongo Main - 192.168.212.173 poort 27020 Mongo Shard1 - ...
0
votes
0answers
13 views

Mongodb does not contain shardkey for pattern

I've created a database with 3 sharded clusters using the wiredTiger engine and I've enable the storage engine option for replicas. Now after enabling sharding for collection, importing a csv to mongo ...
0
votes
1answer
55 views

updating single row in MYSQL

We have a situation where we need to update on a single row several thousands time a second and this requirement lasts continuously for 5 hours every day due to the nature of the business. Design ...
0
votes
0answers
13 views

MySQL 5.6 or 5.7 sharding geospatial data

I have ran into a problem where I need to start to scale horizontally. With my database I am performing only nearest point searches with my data. So from a lat/long inoput I find all the nearest ...
0
votes
0answers
28 views

mysql sharding techniques

I have developed one website for social network and I have used mysql for data storage. Now a days, many tables become a very huge (2-3 tables has 30 million records & 3-4 table has 300k records) ...
4
votes
0answers
43 views

Remove a shard in MongoDB

I have a MongoDB cluster, I accidentally hosed a shard, I don't have a backup. It's pretty easy to recover the data, but after issuing the removeShard command, it says it's draining. However, the ...
0
votes
1answer
68 views

Why sharding does not show any improvement?

Here is what I did to test sharding performance. I have 4 servers, I ran config server and router on a server. 3 others are meant to be shards. First, I only added one shard to router, then I ran a ...
0
votes
1answer
47 views

MongoDB : How to estimate max number of connections supported?

We have a sharded cluster with 2 replica sets. Each replica set is a 3 member cluster with one primary and 2 secondaries. Each instance is a r3.large (aws-ec2) instance with 15G ram. mongos's are in ...
2
votes
2answers
310 views

Mongos version 3.0.1 Split Chunk Error with Sharding

I have 3 config server and two shards. Each shard has a primary, secondary and an arbiter. This is running with the following mongod settings: storageEngine=wiredTiger ...
0
votes
0answers
20 views

MongoDB sharding one collection on single server (2 drives)

I have one single server with two drives (SSD 256GB, HDD 1 TB). What I want to do is to store the newest documents in collection on SSD. Older documents can be stored in HDD drive. Is it possible in ...
0
votes
1answer
58 views

MongoDB Hashed Shard Keys Selection

I have a DB which holds very simple documents called "Item": {_id:<ObjectId>, playerId:int, isRead:bool, content:string} containing about 80M documents. Each Item belongs to a certain player, ...
0
votes
0answers
37 views

Mongo Memory mapping file size issue

I am witnessing some performance issues in my Mongo Sharded cluster. I checked the serverStatus for the memory uses on my shards This is the output Shard one Primary RAM size 512 GB "mem" : { ...
1
vote
1answer
40 views

How to shard a database like that of SO?

I know a bit about sharding but I cannot wrap my head around that fact that say eventually SO cannot scale vertically then how would they shard their SQL DB? Even if we assume SO will not reach that ...
0
votes
0answers
35 views

Issue with total count in sharded collection

I am having a collection which is sharded. This collection receives data from a script which is running hourly. On this data we do some aggregation. My issue when we insert data in collection and run ...
0
votes
1answer
44 views

Is a update-only-once-row table worth sharding?

I have a big MySQL 5.6 Inno DB table that contains user's request. A table row updates only once, shortly-after the initial insert! (The initial insert is to get the unique auto-incrementing ...
0
votes
0answers
29 views

Config data base showing invalid date for collection modification sharding is stopped

For last few days my sharding is not working and every day i am seeing very slow writes. I tried to drop some existing data to create space. But while dropping some old data i got error and when i ...
0
votes
0answers
65 views

Repeat statement of ChunkManager: time to load chunks for empty

In my one of the MongoS server i am continuously getting the below chunkManager statement ChunkManager : time to load chunks for empty This is repeating in logs and the day it started insert in my ...
0
votes
0answers
286 views

Can not drop db in sharded environment

I am trying to delete some old dbs in my sharded set up. Step i am taking are : Stop Balancer. Drop Db Start balancer But in some of the DBs i am getting these errors. "errmsg" : "exception: ...
0
votes
0answers
72 views

Old Mongos servers are found in config database mongo

For some time we are facing issue with our sharded set up as some times one collection randomly stuck on one shard. I have daily db (we create daily database with date) in my application where some ...
0
votes
0answers
88 views

Mongo Chunk 250000 document limit is creating problem in chunk migration

We can set chunk size limit for a DB. But chunk has a limit of 250000 documents per chunk. My chunk size is 250 MB which varies the number of document in each collections. So normally some of my ...
1
vote
1answer
307 views

Mongo db sharded “the collection's metadata lock is taken” issue coming very frequently

I am seeing this warning in mongoS log consistently for one collection. [conn96] warning: splitChunk failed - cmd: { splitChunk: "ibeat20150105.dgrpCount", keyPattern: { articleId: 1, host: 1 }, min: ...
2
votes
1answer
221 views

Chunk Size is shown as 1 KB in mongo db logs even it is set to 300 MB

I am running a mongo cluster. Chunk size set is 300 MB but for today morning it is showing me in logs taht chunk size is 1024 Byte. I checked in current op there also it is showing chunks of 1024 ...
1
vote
2answers
121 views

Help - Error : Could not connect to master of shard replica set

I have a shard, using mongo v2.4.11. It has to mongos, 3 config, and 8 replica set (2 replica for each set, means 16 node for shard) It is keep having exception as per below. Caused by: ...
0
votes
1answer
298 views

I can not copy database MongoDB

I created a mongo cluster in ubuntu and run the copy in the sharding with a small database, everything works fine. The problem is that when I try to copy a database 300gb, is not getting done. The ...
0
votes
1answer
51 views

MongoDB Sharding : What should be the Sharding key if we want to find records with more than one field

I am having collection ("links") with listed fields: collection-name: "links" 1) url 2) title 3) score 4) description 5) ratings (embedded_many) and having index on "url", "title", "score" because ...
1
vote
1answer
146 views

MongoDB Stuck with Jumbo chunks while exhausting a shard

I'm running a shared MongoDB 2.4 cluster. I'm currently in a process of exhausting a shard and I'm down to the last 15 chunks needs to be migrated out of 9660. My issue is that these chunks are too ...
0
votes
1answer
114 views

Choosing shard key with compound index field

I have a compound key with field articleId_1_host_1_timeStamp_1_isIndia_1_location_1. Cardinality of fields is articleId has 20000 different value host has 17 value timestamp is increasing field ...
1
vote
2answers
63 views

Query to Secondary on Sharded Cluster

How to specify query from mongos shell which is inteded to query data from secondary? The goal is not to bother primary load on write and use data in secondary shards instead.
0
votes
1answer
50 views

MongoDB Tag-Aware Sharding Workflow

I wonder the work flow of sharding cluster in Tag-Aware Sharding. --- Sharding Status --- sharding version: { "_id" : 1, "version" : 3, "minCompatibleVersion" : 3, "currentVersion" ...
0
votes
1answer
70 views

MongoDB - Change primary shard only for a certain database

I have a sharded MongoDB cluster running 3 different Databases. Currently all 3 databases has the same shard configured as their primary shard (shard000) Is it possible to change the primary shard for ...
0
votes
1answer
93 views

Repair database on Replica in sharded set up

I am using mongo sharding in my DB. The scenario I am facing is that I have deleted some of my collections in my DB. But as to reclaim the space I need to run repair option which i can not run using ...
0
votes
1answer
422 views

How to update a field in sharded environment in mongodb using C# driver

I have successfully done loading in sharded environment. It is working fine. But when I do update in sharded environment, I am getting the error as : WriteConcern detected an error 'update { q: { ...
0
votes
0answers
20 views

Mongodb sharding balancing not working properly, says config servers differ [duplicate]

Iam using a mongodb sharding configuration in which there are 3 config servers, 2 query routers and 2 shards. One of my shard is having 32 gb data and other is having only 6 gb data. Balancing is not ...
0
votes
0answers
88 views

Problems in sharding in mongodb

I am currently working on sharding. I have certain issues related to sharding : 1.If I load data again to a collection which is already sharded, the data stays only in one shard and not migrating to ...
0
votes
1answer
58 views

Migration of chunks not taking place in sharding

I am trying sharding in MongoDB. I am working with 1 config server 1 Query router 2 shards I tried hash based sharding. The data is getting partitioned but sharding is not successful. In one ...
2
votes
1answer
455 views

mongodb shard chunk migration 500GB takes 13 days - Is this slow or normal?

I have mongodb shard cluster, shard key is hashed. It has 2 shard replica sets. Each replica set has 2 machines. I did an experiment by adding another 2 shard replica sets, and it starts to ...
0
votes
0answers
35 views

Is database sharding done on a table level or database level?

I'm new to sharding databases, so I'm trying to wrap my head around it. Let's say we have a database that has 3 tables: Customer, Suppliers and Orders. Each table has 10 million records. I want to ...
1
vote
1answer
197 views

Is it correct to use IP addresses (instead of hostnames) while configuring Mongo DB cluster? Does it cause any issues?

I have been using Mongo DB Cluster for some data analysis (Mongo DB version 2.4.5). My configuration is as follows: 1) Node1 - mongoconfig server, mongod shard 2) Node2 - mongoconfig server, mongod ...
1
vote
1answer
454 views

Update tag range on MongoDB range shard

Let say I set a min and max key for shards using range tags. Later I find out that the tag range I defined will not balance. Then I have to change the tag range value. I have searched MongoDB's ...
0
votes
0answers
205 views

Slow insert with Mongo 2.6.4 update

Hi I am having an issue with mongo version 2.6.4 my insert queries has became extremely slow after update when i check log on primary i got this 2014-09-16T16:40:18.576+0530 [conn51476] insert ...
0
votes
1answer
129 views

Upgrade lock from mongo 2.4 to 2.6 wont go away

I have upgrade 3 months ago a mongoDB sharded cluster from 2.4.9 to 2.6.1 . I haven't faced any issues with the upgrade and my cluster working fine, but i still see the initial lock that required to ...
0
votes
1answer
326 views

Long running query on replica worker set in mongo

Hi I updated my mongo version from 2.4.to 2.6. Now when i see current op in mongos it shows me some queries which are running for long long time. one example is { "opid" : ...
1
vote
0answers
136 views

Replica failed with error mongo db

I am running mongo sharded cluster with two replica set as shards. One set is working fine but replica secondary on other replica set is not working. When ever i restart it it gives me this error and ...
0
votes
1answer
52 views

Sharded DB accidentally created during update of MongoDB from 2.4.4 to 2.6.4

I am upgrading mongo version 2.4 to 2.6 I was able to upgrade my complete cluster except one primary member of my replica set during this period my auto running script created a sharded db. I can not ...
0
votes
1answer
144 views

Use index intersection feature in mongo 2.6 to compound shard key

I am having three indexes like name,age,salary. I can use them as compound index as per 2.6 (name and age together). But can i use the same indexes as compound shard key like name and age. i also have ...
0
votes
1answer
381 views

Mongo import error: exception:read error, or input line too long (max length: 16777216)

I have a shareded collection which I'm trying to migrate from an existing production Cluster running v 2.4 to a new one running 2.6. As both clusters are shareded, I'm using mongoexport from the old ...
0
votes
2answers
258 views

SUM() the result of a stored procedure

I use the Pl/Proxy extension in order to shard a table. It works like a charm, except for this : I have a stored procedure which runs on 4 shards. It looks like this : CREATE OR REPLACE FUNCTION ...
0
votes
3answers
56 views

Redesign basic Mysql storage architecture

We are a small Internet provider and we are storing all our clients traffic data in a MySQL database (MyISAM). We insert data every few minutes in three tables like this: capture_table ---------- ...
1
vote
1answer
213 views

Should all machines which run mongo instances (mongod or mongos) in a Mongo DB cluster have unique hostnames (also at OS level)?

I have been using Mongo DB Cluster for some data analysis (Mongo DB version 2.4.5) and while inserting some data into the cluster, I found an error which is the same as mentioned in the bug : ...