Sharding is the practice of logically dividing or partitioning data, usually using a sprecific key (referred to as a shard key), and then placing that data on separate hosts (subsequently known as shards). This is generally done to scale horizontally (more hosts) as opposed to vertically (more ...

learn more… | top users | synonyms

1
vote
1answer
31 views

MongoDB Stuck with Jumbo chunks while exhausting a shard

I'm running a shared MongoDB 2.4 cluster. I'm currently in a process of exhausting a shard and I'm down to the last 15 chunks needs to be migrated out of 9660. My issue is that these chunks are too ...
0
votes
1answer
33 views

Choosing shard key with compound index field

I have a compound key with field articleId_1_host_1_timeStamp_1_isIndia_1_location_1. Cardinality of fields is articleId has 20000 different value host has 17 value timestamp is increasing field ...
1
vote
2answers
20 views

Query to Secondary on Sharded Cluster

How to specify query from mongos shell which is inteded to query data from secondary? The goal is not to bother primary load on write and use data in secondary shards instead.
0
votes
1answer
16 views

MongoDB Tag-Aware Sharding Workflow

I wonder the work flow of sharding cluster in Tag-Aware Sharding. --- Sharding Status --- sharding version: { "_id" : 1, "version" : 3, "minCompatibleVersion" : 3, "currentVersion" ...
0
votes
1answer
22 views

MongoDB - Change primary shard only for a certain database

I have a sharded MongoDB cluster running 3 different Databases. Currently all 3 databases has the same shard configured as their primary shard (shard000) Is it possible to change the primary shard for ...
0
votes
1answer
32 views

Repair database on Replica in sharded set up

I am using mongo sharding in my DB. The scenario I am facing is that I have deleted some of my collections in my DB. But as to reclaim the space I need to run repair option which i can not run using ...
0
votes
1answer
49 views

How to update a field in sharded environment in mongodb using C# driver

I have successfully done loading in sharded environment. It is working fine. But when I do update in sharded environment, I am getting the error as : WriteConcern detected an error 'update { q: { ...
0
votes
0answers
20 views

Mongodb sharding balancing not working properly, says config servers differ [duplicate]

Iam using a mongodb sharding configuration in which there are 3 config servers, 2 query routers and 2 shards. One of my shard is having 32 gb data and other is having only 6 gb data. Balancing is not ...
0
votes
0answers
61 views

Problems in sharding in mongodb

I am currently working on sharding. I have certain issues related to sharding : 1.If I load data again to a collection which is already sharded, the data stays only in one shard and not migrating to ...
0
votes
1answer
31 views

Migration of chunks not taking place in sharding

I am trying sharding in MongoDB. I am working with 1 config server 1 Query router 2 shards I tried hash based sharding. The data is getting partitioned but sharding is not successful. In one ...
1
vote
1answer
70 views

mongodb shard chunk migration 500GB takes 13 days - Is this slow or normal?

I have mongodb shard cluster, shard key is hashed. It has 2 shard replica sets. Each replica set has 2 machines. I did an experiment by adding another 2 shard replica sets, and it starts to ...
0
votes
0answers
21 views

Is database sharding done on a table level or database level?

I'm new to sharding databases, so I'm trying to wrap my head around it. Let's say we have a database that has 3 tables: Customer, Suppliers and Orders. Each table has 10 million records. I want to ...
0
votes
1answer
42 views

Is it correct to use IP addresses (instead of hostnames) while configuring Mongo DB cluster? Does it cause any issues?

I have been using Mongo DB Cluster for some data analysis (Mongo DB version 2.4.5). My configuration is as follows: 1) Node1 - mongoconfig server, mongod shard 2) Node2 - mongoconfig server, mongod ...
1
vote
1answer
95 views

Update tag range on MongoDB range shard

Let say I set a min and max key for shards using range tags. Later I find out that the tag range I defined will not balance. Then I have to change the tag range value. I have searched MongoDB's ...
0
votes
0answers
119 views

Slow insert with Mongo 2.6.4 update

Hi I am having an issue with mongo version 2.6.4 my insert queries has became extremely slow after update when i check log on primary i got this 2014-09-16T16:40:18.576+0530 [conn51476] insert ...
0
votes
1answer
51 views

Upgrade lock from mongo 2.4 to 2.6 wont go away

I have upgrade 3 months ago a mongoDB sharded cluster from 2.4.9 to 2.6.1 . I haven't faced any issues with the upgrade and my cluster working fine, but i still see the initial lock that required to ...
0
votes
1answer
74 views

Long running query on replica worker set in mongo

Hi I updated my mongo version from 2.4.to 2.6. Now when i see current op in mongos it shows me some queries which are running for long long time. one example is { "opid" : ...
1
vote
0answers
57 views

Replica failed with error mongo db

I am running mongo sharded cluster with two replica set as shards. One set is working fine but replica secondary on other replica set is not working. When ever i restart it it gives me this error and ...
0
votes
1answer
32 views

Sharded DB accidentally created during update of MongoDB from 2.4.4 to 2.6.4

I am upgrading mongo version 2.4 to 2.6 I was able to upgrade my complete cluster except one primary member of my replica set during this period my auto running script created a sharded db. I can not ...
0
votes
1answer
86 views

Use index intersection feature in mongo 2.6 to compound shard key

I am having three indexes like name,age,salary. I can use them as compound index as per 2.6 (name and age together). But can i use the same indexes as compound shard key like name and age. i also have ...
0
votes
1answer
159 views

Mongo import error: exception:read error, or input line too long (max length: 16777216)

I have a shareded collection which I'm trying to migrate from an existing production Cluster running v 2.4 to a new one running 2.6. As both clusters are shareded, I'm using mongoexport from the old ...
0
votes
2answers
100 views

SUM() the result of a stored procedure

I use the Pl/Proxy extension in order to shard a table. It works like a charm, except for this : I have a stored procedure which runs on 4 shards. It looks like this : CREATE OR REPLACE FUNCTION ...
0
votes
3answers
51 views

Redesign basic Mysql storage architecture

We are a small Internet provider and we are storing all our clients traffic data in a MySQL database (MyISAM). We insert data every few minutes in three tables like this: capture_table ---------- ...
1
vote
1answer
72 views

Should all machines which run mongo instances (mongod or mongos) in a Mongo DB cluster have unique hostnames (also at OS level)?

I have been using Mongo DB Cluster for some data analysis (Mongo DB version 2.4.5) and while inserting some data into the cluster, I found an error which is the same as mentioned in the bug : ...
0
votes
1answer
63 views

Updating mongo sharded cluster without down time for mongod

I am trying to update my mongo sharded version from 2.4 to 2.6.3. I have some queries on steps to update this. When I will run one mongos with the --upgrade option, will it update all the config ...
0
votes
1answer
223 views

Mongo large chunks will not split

I had a 3 shard set up, and ran out of capacity, so I added 3 more shards. (Each shard is a replica set). But the data isn't spread across the cluster evenly. I have my chunkSize set to the standard ...
1
vote
1answer
201 views

Mongo sharding issue with chunk split and Data transfer

I am getting this error in my primary shard warning: moveChunk failed to engage TO-shard in the data transfer: still waiting for a previous migrates data to get cleaned, can't accept new chunks, num ...
1
vote
1answer
66 views

MongoDB index strategy

I'm developing a system to back-up computers and store the files / data with retention for a while. The actual files are stored in an object store, while I'm planning to keep the metadata of the files ...
0
votes
1answer
28 views

How to confirm that chunk transfer is completed on a DB

I have one case where i create db for Daily data. I have some collections in this db which are sharded. My requirement is as follows: Create every day DB If There is a DB which is more than 7 days old ...
1
vote
0answers
200 views

Error to connect mongos when trying to create replicated sharded cluster

I'm trying to create an replicated sharded cluster in mongodb. Initially I've created two shards and there are a replica set with three members in each shard. And all the shards and replicasets run in ...
0
votes
0answers
47 views

Sharding TPC-C Benchmark with MySQL Fabric

Currently I have a setup of 5 mysql database nodes (db01 ~ db05). MySQL Fabric and backstore have been installed and running in db01. I configured one global_group (i.e. group_02) with db02 and three ...
1
vote
1answer
228 views

MongoDB Shard - Disable Cursor timeout

I'm using an sharded-cluster(Mongo version 2.4.1) with a java application. After a specific time (10 minutes or more) I got the following exception: com.mongodb.MongoException$Network: Read operation ...
0
votes
1answer
88 views

how to split the chunks in sharding

I have edited the question. I was trying to run the sh.splitAt mongos> sh.splitAt("test.zp",{"city": "1"})it worked. But the problem is at sharding chunks. when i have run the command ...
2
votes
0answers
202 views

Mariadb sharding?

I'm trying to implement a mariadb sharding for learning purpose. I have looked into database sharding in general and found mariadb spider storage engine support sharding. I do not want spider storage ...
0
votes
0answers
39 views

Would an INNER JOIN query in a sharded relational database hit the same shard when using a compound primary key?

In this question I have asked about modulus calculation in sharded tables, this design is used: CREATE TABLE Orders (order_id INT PRIMARY KEY, ...); CREATE TABLE LineItems (order_id INT, lineitem_id ...
1
vote
0answers
36 views

Modulus calculation for compound primary keys in MySQL Cluster (and other sharded databases)

In MySQL Cluster, and generally in sharded databases, how is a compound primary key's modulus calculated? Say, for example, there are two tables that have primary keys like so: CREATE TABLE Orders ...
0
votes
1answer
88 views

MongoDB - temporarily turn shard to standalone for local user creation

Following question Mongodb sharded cluster backup user issue I have a sharded mongodb cluster, with 4 shards, and I'm required to stop the mongod service on each shard, and start it not as a part of ...
3
votes
1answer
272 views

MongoDB presplitting chunks for compound shard key

In my mongodb setup I have a compound shard key {"region" : 1, "foo" : 1, "bar" : 1} and I know the values region can be and that each region should be on one chunk. Therefore I'd like to pre-split ...
1
vote
1answer
70 views

Mongodb sharded cluster backup user issue

We're using a simple python script to backup our mongodb cluster. the script connects to each mongod instance via ssh, and performs mongodump, all without requiring user/pass ever since we started ...
0
votes
1answer
171 views

MongoDB: The collections metadata lock is taken

Running a mongodb 2.6 cluster on localhost with four shards and four replication sets. Using a for loop with 100 iterations that pre-splits chunks among the existing shards. I am getting the error ...
0
votes
1answer
53 views

MongoDB only three of four shards being used

Using MongoDB 2.6, I've setup four replication sets: a, b, c & d: >rs.conf() a:PRIMARY> rs.conf() { "_id" : "name_of_replset", "version" : 1, "members" : [ { ...
1
vote
1answer
244 views

Mongodb: slow shard balancing

I've decided to shard a collection with 500M+ documents into 24 shards. All shards are on the same machine without replicas as was advised in this title (core sharding). The shard key is {"_id":1}. I ...
0
votes
1answer
79 views

Oracle DB - Is it possible to distribute data of a table onto multiple servers?

BigTable[1]-like systems (HBase[2] for instance) allows a table to be distributed transparently accross multiple servers. This works as follows: Each server perform the read/writes operations on ...
0
votes
1answer
77 views

MongoDB - sharding

I have a sharded cluster with 4 shards, 3 config databases. My problem is when I do my inserts only one shard gets filled. Can anyone please explain to me why this is happening. My first thought was ...
2
votes
1answer
171 views

Scaling out SQL Server and syncing data across multiple machines

I don't have expertise in architecting databases, and I've been teaching myself new stuff every day. I'd like to make an Internet-scale application using SQL Server as the data store. I haven't found ...
0
votes
1answer
135 views

Split Vector is taking too much time in mongo shard

I am using mongo shard distribution system. This system performs slow as the data size increases. I checked with profiler and saw split vector is taking to much time while running. It happened after ...
0
votes
0answers
57 views

PostgreSQL manual way of sharding like behaviour

I've a main server with low space available and good internet connection that has a postgres database in production for a zabbix system. Also, I've another secundary server plenty of space but with ...
0
votes
0answers
16 views

Index table for horizontal shard

Lets assume we have a vary amount of clients, which could manage some stuff on our site. Each client has his own table, which is used to store data. For instance: client_1, client_2, ..., client_N. ...
1
vote
1answer
319 views

Removing Database from sharded cluster in Mongo DB

I Have a mongo shard db system with two shards and and three mongos servers.My application creates DB on daily basis as per our business requirements. We have some sharded collection in each day DB. ...
0
votes
0answers
424 views

Mongodb: Waited too long for lock balancer to unlock

Mongo Version: 2.4.5 OS: Windows Server 2008 R2 Standard (Service Pack 1) We have a balancer idling for a day doing nothing (no recent records in changelog), holding a lock state 2. Tried to stop ...