1

Is it possible to replicate a data item across multiple shards?

My end goal is to determine how up-to-date the data in a replica slave is. I'm only concerned with subsets of the data, so I don't think examining oplogs will work. Instead, I plan on having a counter increment in the master, every time a modification to the appropriate data occurs. Then, I can fetch the slave's replicated counter to see how current it is.

The problem occurs when I try to shard the data. I need the counter replicated across every replica, not just the replica set that owns the counter data item. So, I need some way to replicate the counter to every master shard, where it will then be replicated to each slave. Is this possible?

Modifications to the counter would still go through the master shard that "owns" the counter. Data reads would need to aggregate the counter value from each shard/replica, taking the minimum value. I read through the MongoDB documentation, and I really don't think it's possible. But perhaps one of you MongoDB experts/developers can shed some hope.

Someone on SO mentioned replSetGetStatus, which may be an option. However, I would like separate counters for different subsets of the data, where the replica status corresponds to all the data.

0
  1. Create 'c' -database
  2. Add c -collection
  3. Shard this collection with _id
  4. Set this collection balacing:false
  5. Pre-split this collection with _id values: 0,1,2,3,.. (or your real shard names). How many shards you have; one per shard
  6. With moveChunk command, move one chunk to every shard, so that right _id is on right shard.

Now if you update this c.c's counter-key with $inc -operator (what is atomic operation) and without _id is set but multi:true, you can always "query" min value of that counter-key and answer tells what shard and what was value.

db.getSiblingDB('c').c.update({},{$inc:{"counter":1}},{multi:true})

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.