0

I'm not sure if this question been asked before or if the following behavior of MongoDB is normal. Searching online output no results to this scenario.

Initially, we had a 3 node deployment, 1 Primary, 1 Secondary, and 1 Arbiter.

We wanted to add a ReadOnly replica to the cluster and remove the Arbiter node as well in the process. We added the following to the new node:

priority: 0 hidden: true votes: 1 And removed the Arbiter in the same reconfiguration process so we always have 3 voting members and it leaves us with 1 Primary and 1 Secondary and 1 ReadOnly Node.

The complete process went through smoothly, however, we still end up seeing connections to the ReadOnly replica.

But when checking via db.currentOp(), no queries show up.

Based on the documentation on MongoDB website,

Hidden members are part of a replica set but cannot become primary and are invisible to client applications.

Is there a way to investigate why there are connections coming in? And if this is normal behavior?

EDIT: (for further clarification)

Assuming the following:

MongoDB A (Primary): 192.168.1.50

MongoDB B (Secondary): 192.168.1.51

MongoDB C (Hidden): 192.168.1.52

Client A: 192.168.1.60

Client B: 192.168.1.61

In the logs, we see the following:

2018-03-12T07:19:11.607+0000 I ACCESS   [conn119719] Successfully authenticated as principal SOMEUSER on SOMEDB
2018-03-12T07:19:11.607+0000 I NETWORK  [conn119719] end connection 192.168.1.60 (2 connections now open)
2018-03-12T07:19:17.087+0000 I NETWORK  [listener] connection accepted from 192.168.1.60:47806 #119720 (3 connections now open)
2018-03-12T07:19:17.371+0000 I ACCESS   [conn119720] Successfully authenticated as principal SOMEUSER on SOMEDB

So if the other MongoDB instances were connecting, that would be fine, but my question is regarding why the clients are able to connect even when the hidden option is true and if that behavior is normal.

EDIT2:

To further clarify, the connection string does not include MongoDB C.

Connection String Format:

mongodb://mongodb-a.domain:27017,mongodb-b.domain:27017/?replicaSet=MyReplica

Hence, the client does not know MongoDB C exists and by theory, it should be hidden, but in our case it's not.

  • ,currentOp must run against the admin database, and it can accept several optional fields. The currentOp() Returns a document that contains information on in-progress operations for the mongod instance. – Md Haidar Ali Khan Mar 12 '18 at 6:53
  • Outstanding questions from your duplicate post on StackOverflow (stackoverflow.com/questions/49186717/…): What specific MongoDB driver and version are you using? Is the SOMEUSER noted in the log definitely a user from your application and not a manual connection? Are these connections always closed immediately after successful authentication? One more thought: would the hidden secondary happen to be on the same IP address and port as the former arbiter? – Stennie Mar 12 '18 at 16:28
1

Yes, it is normal behavior. Those connections are from other mongod nodes, because every node "watches" every other node to see who is reachable. This because election must be started even in the situation where secondary node disappear from replica set.

  • Added an edit section to clarify further on what I'm trying to figure out – Nikhil C. Mar 12 '18 at 7:25
  • Have you checked that 192.168.1.60 connection string covers all nodes at replica set and not just this hidden node? Because direct connections are allowed to hidden nodes (for reporting purpose f.ex.) – JJussi Mar 12 '18 at 8:00
  • The config string does not include this node at all. It includes the A and B, and C has just been added and configured to be a Hidden node so in theory, the client should not even know that C exists, yet connections still come to C. – Nikhil C. Mar 12 '18 at 8:08
  • @Nikhil C., As JJussi, alreday said that 192.168.1.60 connection string covers all nodes at replica set and not just this hidden node. – Md Haidar Ali Khan Mar 12 '18 at 8:14
  • @MdHaidarAliKhan added another edit to the question explaining this. The client does not know of MongoDB C. In the connection string we only provide connection to A and B, yet connections still go to C. – Nikhil C. Mar 12 '18 at 8:28
1

The purpose of a hidden node is to provide specific read-only workloads, or have copies over your data. which are hidden from the application.Hidden nodes can also be set with a delay in their replication process. We call these delayed nodes.

The purpose of having delayed nodes is to allow resilience to application level corruption,without relying on cold backup files to recover from such an event. If we have a node delayed let's say one hour, and if your DBA accidentally drops a collection, we have one hour to recover all the data from the delayed node without needing to go back to back up file to recover to whatever the time that backup was created.

A hidden member maintains a copy of the primary’s data set but is invisible to client applications. Hidden members are good for workloads with different usage patterns from the other members in the replica set. Hidden members must always be priority 0 members and so cannot become primary. The db.isMaster() method does not display hidden members. Hidden members, however, may vote in elections. The most common use of hidden nodes is to support delayed members.

Read Operations: Clients will not distribute reads with the appropriate read preference to hidden members. As a result, these members receive no traffic other than basic replication. Use hidden members for dedicated tasks such as reporting and backups.

Delayed members should be hidden. In a sharded cluster, mongos do not interact with hidden members.

Voting: Hidden members may vote in replica set elections.If you stop a voting hidden member, ensure that the set has an active majority or the primary will step down.

As per MongoDB BOL Documentation here The most common use of hidden nodes is to support delayed members. If you only need to prevent a member from becoming primary, configure a priority 0 member.

As MongoDB documentation here for consideration requirements of Delayed members:

  1. Must be priority 0 members. Set the priority to 0 to prevent a delayed member from becoming primary.
  2. Should be hidden members. Always prevent applications from seeing and querying delayed members.
  3. do vote in elections for primary, if members[n].votes is set to 1.

Note: A priority 0 member is a member that cannot become primary and cannot trigger elections.

For further your ref Here and Here

  • 1
    That doesn't really answer the question, since that information is already known and hence trying to figure out why client connections are still coming to this instance – Nikhil C. Mar 12 '18 at 7:24
  • @Nikhil C.,The most common use of hidden nodes is to support delayed members. If you only need to prevent a member from becoming primary, configure a priority 0 member. As JJussi has already said that . – Md Haidar Ali Khan Mar 12 '18 at 7:50

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.