I have a MongoDB cluster setup with one primary and 2 Secondaries in a single shard setup. Every night we have a high load but one day and most of the read APIs have a SECONDARY_PREFERRED read preference.

But the load not balanced as shown below: enter image description here

X-Axis -> time Y-Axis -> number of queries

Green -> Secondary one Yellow -> Secondary two

And the CPU of the corresponding machines enter image description here

X-Axis -> time Y-Axis -> % of CPU IDLE

Green -> Secondary one Yellow -> Secondary two

We are using Mongo Java Driver Why would the query be distributed unevenly?


The algorithm used by MongoDB drivers to select a node is detailed at https://docs.mongodb.com/manual/core/read-preference-mechanics/#server-selection-algorithm.

For secondaryPreferred it will:

  • compile a list of secondary nodes that meet both maxStalenessSeconds and and tag sets
  • limit the list to those with network latency within localThresholdMS (default 15ms)
  • randomly select one of the nodes that remain

So if a node experiences some replication lag, or a network glitch makes the packets to one secondary slower than the other, or if the random number generator doesn't generate sufficiently uniform values, there will be more connections to one secondary than the other.

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.