0

I have two question which are rally about the same thing - how to read more documents per second? I'm running a MongoDB cluster on Amazon EC2 and I'm load testing it from 10 servers (on Amazon EMR). It looks like I hit Amazon's bandwidth cap long before hardware issues.

MongoDB cluster

  • Mongo 3.2.4
  • 6x EC2 r3.2xlarge
  • 3x shard 2x replicas
  • Data:

Totals data : 36.91GiB docs : 7183252 chunks : 1024 Shard shard0 contains 31.24% data, 31.23% docs in cluster, avg obj size on shard : 5KiB Shard shard1 contains 32.48% data, 32.48% docs in cluster, avg obj size on shard : 5KiB Shard shard2 contains 36.27% data, 36.28% docs in cluster, avg obj size on shard : 5KiB

Load Test Environment

  • Amazon EMR (the same availability zone)
  • 11x EC2 m3.xlarge
  • Local instance of Mongos connected to the MongoDB cluster

Problem 1

I would like client to use all available server (Primary and Secondary). From my understanding I have to use ReadPreference NEAREST:

clinet = new MongoClient(Arrays.asList( new ServerAddress("ip-172-30-0-98", 27017))); clinet.setReadPreference(ReadPreference.nearest());

Unfortunately this doesn't seam to be working. When I send lots of queries to the cluster only PRIMARY servers look busy.

High load only on PRIMARY servers

In this test I exclude the biggies filed (Doc) which is ~5Kb. It allows me to read 250,000 documents per second.

collection.find(in(keyName, ids)).projection(Projections.exclude("Doc")).into(documents);

It looks to me like I'm wasting 50% of potential capacity of my cluster. What should I do to utilise it fully?

Problem 2

When I load the whole document collection.find(in(keyName, ids)).into(documents) I can only read ~22,000 documents per second. Quick look into htop shows light load on the cluster.

low cluster utilisation

The only explanation to this doc/sec drop I can find is Network I/O. If an average document is 5kb and I'm reading 22k per second does it mean that my Mongos instance can only read ~107MB per second from the cluster?

If I didn't miss anything obvious how should this problem be solve? Add more Mongos?

migrated from stackoverflow.com Apr 5 '16 at 0:03

This question came from our site for professional and enthusiast programmers.

  • 1
    Regarding your first question. nearest means that client will connect whichever node with less network latency. Since your servers are co-located in probably same DC, primary will be selected most of time. If you want best performance and offload primary, use secondary preferred. – Saleem Apr 4 '16 at 16:36
  • 1
    How many mongos are you running? Also your replica-set is quite odd. Having 2 member replica-set you'll not get any automatic fail-over. – Saleem Apr 6 '16 at 12:52

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy