Sharding is the practice of logically dividing or partitioning data, usually using a sprecific key (referred to as a shard key), and then placing that data on separate hosts (subsequently known as shards). This is generally done to scale horizontally (more hosts) as opposed to vertically (more ...

learn more… | top users | synonyms

0
votes
0answers
10 views

Mongo Memory mapping file size issue

I am witnessing some performance issues in my Mongo Sharded cluster. I checked the serverStatus for the memory uses on my shards This is the output Shard one Primary RAM size 512 GB "mem" : { ...
1
vote
1answer
34 views

How to shard a database like that of SO?

I know a bit about sharding but I cannot wrap my head around that fact that say eventually SO cannot scale vertically then how would they shard their SQL DB? Even if we assume SO will not reach that ...
0
votes
0answers
25 views

Issue with total count in sharded collection

I am having a collection which is sharded. This collection receives data from a script which is running hourly. On this data we do some aggregation. My issue when we insert data in collection and run ...
0
votes
0answers
19 views

Is a update-only-once-row table worth sharding?

I have a big MySQL 5.6 Inno DB table that contains user's request. A table row updates only once, shortly-after the initial insert! (The initial insert is to get the unique auto-incrementing ...
0
votes
0answers
17 views

Config data base showing invalid date for collection modification sharding is stopped

For last few days my sharding is not working and every day i am seeing very slow writes. I tried to drop some existing data to create space. But while dropping some old data i got error and when i ...
0
votes
0answers
19 views

Repeat statement of ChunkManager: time to load chunks for empty

In my one of the MongoS server i am continuously getting the below chunkManager statement ChunkManager : time to load chunks for empty This is repeating in logs and the day it started insert in my ...
0
votes
0answers
45 views

Can not drop db in sharded environment

I am trying to delete some old dbs in my sharded set up. Step i am taking are : Stop Balancer. Drop Db Start balancer But in some of the DBs i am getting these errors. "errmsg" : "exception: ...
0
votes
0answers
27 views

Old Mongos servers are found in config database mongo

For some time we are facing issue with our sharded set up as some times one collection randomly stuck on one shard. I have daily db (we create daily database with date) in my application where some ...
0
votes
0answers
37 views

Mongo Chunk 250000 document limit is creating problem in chunk migration

We can set chunk size limit for a DB. But chunk has a limit of 250000 documents per chunk. My chunk size is 250 MB which varies the number of document in each collections. So normally some of my ...
0
votes
1answer
52 views

Mongo db sharded “the collection's metadata lock is taken” issue coming very frequently

I am seeing this warning in mongoS log consistently for one collection. [conn96] warning: splitChunk failed - cmd: { splitChunk: "ibeat20150105.dgrpCount", keyPattern: { articleId: 1, host: 1 }, min: ...
1
vote
1answer
105 views

Chunk Size is shown as 1 KB in mongo db logs even it is set to 300 MB

I am running a mongo cluster. Chunk size set is 300 MB but for today morning it is showing me in logs taht chunk size is 1024 Byte. I checked in current op there also it is showing chunks of 1024 ...
1
vote
1answer
28 views

Help - Error : Could not connect to master of shard replica set

I have a shard, using mongo v2.4.11. It has to mongos, 3 config, and 8 replica set (2 replica for each set, means 16 node for shard) It is keep having exception as per below. Caused by: ...
0
votes
1answer
63 views

I can not copy database MongoDB

I created a mongo cluster in ubuntu and run the copy in the sharding with a small database, everything works fine. The problem is that when I try to copy a database 300gb, is not getting done. The ...
0
votes
1answer
28 views

MongoDB Sharding : What should be the Sharding key if we want to find records with more than one field

I am having collection ("links") with listed fields: collection-name: "links" 1) url 2) title 3) score 4) description 5) ratings (embedded_many) and having index on "url", "title", "score" because ...
1
vote
1answer
48 views

MongoDB Stuck with Jumbo chunks while exhausting a shard

I'm running a shared MongoDB 2.4 cluster. I'm currently in a process of exhausting a shard and I'm down to the last 15 chunks needs to be migrated out of 9660. My issue is that these chunks are too ...
0
votes
1answer
44 views

Choosing shard key with compound index field

I have a compound key with field articleId_1_host_1_timeStamp_1_isIndia_1_location_1. Cardinality of fields is articleId has 20000 different value host has 17 value timestamp is increasing field ...
1
vote
2answers
27 views

Query to Secondary on Sharded Cluster

How to specify query from mongos shell which is inteded to query data from secondary? The goal is not to bother primary load on write and use data in secondary shards instead.
0
votes
1answer
24 views

MongoDB Tag-Aware Sharding Workflow

I wonder the work flow of sharding cluster in Tag-Aware Sharding. --- Sharding Status --- sharding version: { "_id" : 1, "version" : 3, "minCompatibleVersion" : 3, "currentVersion" ...
0
votes
1answer
27 views

MongoDB - Change primary shard only for a certain database

I have a sharded MongoDB cluster running 3 different Databases. Currently all 3 databases has the same shard configured as their primary shard (shard000) Is it possible to change the primary shard for ...
0
votes
1answer
40 views

Repair database on Replica in sharded set up

I am using mongo sharding in my DB. The scenario I am facing is that I have deleted some of my collections in my DB. But as to reclaim the space I need to run repair option which i can not run using ...
0
votes
1answer
97 views

How to update a field in sharded environment in mongodb using C# driver

I have successfully done loading in sharded environment. It is working fine. But when I do update in sharded environment, I am getting the error as : WriteConcern detected an error 'update { q: { ...
0
votes
0answers
20 views

Mongodb sharding balancing not working properly, says config servers differ [duplicate]

Iam using a mongodb sharding configuration in which there are 3 config servers, 2 query routers and 2 shards. One of my shard is having 32 gb data and other is having only 6 gb data. Balancing is not ...
0
votes
0answers
65 views

Problems in sharding in mongodb

I am currently working on sharding. I have certain issues related to sharding : 1.If I load data again to a collection which is already sharded, the data stays only in one shard and not migrating to ...
0
votes
1answer
34 views

Migration of chunks not taking place in sharding

I am trying sharding in MongoDB. I am working with 1 config server 1 Query router 2 shards I tried hash based sharding. The data is getting partitioned but sharding is not successful. In one ...
1
vote
1answer
112 views

mongodb shard chunk migration 500GB takes 13 days - Is this slow or normal?

I have mongodb shard cluster, shard key is hashed. It has 2 shard replica sets. Each replica set has 2 machines. I did an experiment by adding another 2 shard replica sets, and it starts to ...
0
votes
0answers
21 views

Is database sharding done on a table level or database level?

I'm new to sharding databases, so I'm trying to wrap my head around it. Let's say we have a database that has 3 tables: Customer, Suppliers and Orders. Each table has 10 million records. I want to ...
0
votes
1answer
56 views

Is it correct to use IP addresses (instead of hostnames) while configuring Mongo DB cluster? Does it cause any issues?

I have been using Mongo DB Cluster for some data analysis (Mongo DB version 2.4.5). My configuration is as follows: 1) Node1 - mongoconfig server, mongod shard 2) Node2 - mongoconfig server, mongod ...
1
vote
1answer
120 views

Update tag range on MongoDB range shard

Let say I set a min and max key for shards using range tags. Later I find out that the tag range I defined will not balance. Then I have to change the tag range value. I have searched MongoDB's ...
0
votes
0answers
148 views

Slow insert with Mongo 2.6.4 update

Hi I am having an issue with mongo version 2.6.4 my insert queries has became extremely slow after update when i check log on primary i got this 2014-09-16T16:40:18.576+0530 [conn51476] insert ...
0
votes
1answer
60 views

Upgrade lock from mongo 2.4 to 2.6 wont go away

I have upgrade 3 months ago a mongoDB sharded cluster from 2.4.9 to 2.6.1 . I haven't faced any issues with the upgrade and my cluster working fine, but i still see the initial lock that required to ...
0
votes
1answer
110 views

Long running query on replica worker set in mongo

Hi I updated my mongo version from 2.4.to 2.6. Now when i see current op in mongos it shows me some queries which are running for long long time. one example is { "opid" : ...
1
vote
0answers
70 views

Replica failed with error mongo db

I am running mongo sharded cluster with two replica set as shards. One set is working fine but replica secondary on other replica set is not working. When ever i restart it it gives me this error and ...
0
votes
1answer
35 views

Sharded DB accidentally created during update of MongoDB from 2.4.4 to 2.6.4

I am upgrading mongo version 2.4 to 2.6 I was able to upgrade my complete cluster except one primary member of my replica set during this period my auto running script created a sharded db. I can not ...
0
votes
1answer
98 views

Use index intersection feature in mongo 2.6 to compound shard key

I am having three indexes like name,age,salary. I can use them as compound index as per 2.6 (name and age together). But can i use the same indexes as compound shard key like name and age. i also have ...
0
votes
1answer
204 views

Mongo import error: exception:read error, or input line too long (max length: 16777216)

I have a shareded collection which I'm trying to migrate from an existing production Cluster running v 2.4 to a new one running 2.6. As both clusters are shareded, I'm using mongoexport from the old ...
0
votes
2answers
117 views

SUM() the result of a stored procedure

I use the Pl/Proxy extension in order to shard a table. It works like a charm, except for this : I have a stored procedure which runs on 4 shards. It looks like this : CREATE OR REPLACE FUNCTION ...
0
votes
3answers
53 views

Redesign basic Mysql storage architecture

We are a small Internet provider and we are storing all our clients traffic data in a MySQL database (MyISAM). We insert data every few minutes in three tables like this: capture_table ---------- ...
1
vote
1answer
94 views

Should all machines which run mongo instances (mongod or mongos) in a Mongo DB cluster have unique hostnames (also at OS level)?

I have been using Mongo DB Cluster for some data analysis (Mongo DB version 2.4.5) and while inserting some data into the cluster, I found an error which is the same as mentioned in the bug : ...
0
votes
1answer
68 views

Updating mongo sharded cluster without down time for mongod

I am trying to update my mongo sharded version from 2.4 to 2.6.3. I have some queries on steps to update this. When I will run one mongos with the --upgrade option, will it update all the config ...
0
votes
1answer
294 views

Mongo large chunks will not split

I had a 3 shard set up, and ran out of capacity, so I added 3 more shards. (Each shard is a replica set). But the data isn't spread across the cluster evenly. I have my chunkSize set to the standard ...
1
vote
1answer
232 views

Mongo sharding issue with chunk split and Data transfer

I am getting this error in my primary shard warning: moveChunk failed to engage TO-shard in the data transfer: still waiting for a previous migrates data to get cleaned, can't accept new chunks, num ...
1
vote
1answer
70 views

MongoDB index strategy

I'm developing a system to back-up computers and store the files / data with retention for a while. The actual files are stored in an object store, while I'm planning to keep the metadata of the files ...
0
votes
1answer
35 views

How to confirm that chunk transfer is completed on a DB

I have one case where i create db for Daily data. I have some collections in this db which are sharded. My requirement is as follows: Create every day DB If There is a DB which is more than 7 days old ...
1
vote
0answers
222 views

Error to connect mongos when trying to create replicated sharded cluster

I'm trying to create an replicated sharded cluster in mongodb. Initially I've created two shards and there are a replica set with three members in each shard. And all the shards and replicasets run in ...
0
votes
0answers
48 views

Sharding TPC-C Benchmark with MySQL Fabric

Currently I have a setup of 5 mysql database nodes (db01 ~ db05). MySQL Fabric and backstore have been installed and running in db01. I configured one global_group (i.e. group_02) with db02 and three ...
1
vote
1answer
309 views

MongoDB Shard - Disable Cursor timeout

I'm using an sharded-cluster(Mongo version 2.4.1) with a java application. After a specific time (10 minutes or more) I got the following exception: com.mongodb.MongoException$Network: Read operation ...
0
votes
1answer
94 views

how to split the chunks in sharding

I have edited the question. I was trying to run the sh.splitAt mongos> sh.splitAt("test.zp",{"city": "1"})it worked. But the problem is at sharding chunks. when i have run the command ...
2
votes
0answers
211 views

Mariadb sharding?

I'm trying to implement a mariadb sharding for learning purpose. I have looked into database sharding in general and found mariadb spider storage engine support sharding. I do not want spider storage ...
0
votes
0answers
42 views

Would an INNER JOIN query in a sharded relational database hit the same shard when using a compound primary key?

In this question I have asked about modulus calculation in sharded tables, this design is used: CREATE TABLE Orders (order_id INT PRIMARY KEY, ...); CREATE TABLE LineItems (order_id INT, lineitem_id ...
1
vote
0answers
39 views

Modulus calculation for compound primary keys in MySQL Cluster (and other sharded databases)

In MySQL Cluster, and generally in sharded databases, how is a compound primary key's modulus calculated? Say, for example, there are two tables that have primary keys like so: CREATE TABLE Orders ...