6

I'm using MongoDB version 3.0.0. I'm trying to setup mongodb replication on our machine. Replication was initially setup but due to some changes on the VM the entire thing crashed. When I tried to set it up again, the secondary and Arbiter became stuck in StartUp Mode.

In the mongoDB conf file I have set

replSet=ReplicaSet1

I added 2 machines using the command

rs.add("10.235.96.12:27017")
rs.add("10.235.96.12:27017")

But after this on the primary when I do a rs.status() secondary and arbiter are still shown in StartUp

ReplicaSet1:PRIMARY> rs.status()
{
    "set" : "ReplicaSet1",
    "date" : ISODate("2015-07-31T04:45:57.260Z"),
    "myState" : 1,
    "members" : [
            {
                    "_id" : 0,
                    "name" : "BOSPROD9:27017",
                    "health" : 1,
                    "state" : 1,
                    "stateStr" : "PRIMARY",
                    "uptime" : 63104,
                    "optime" : Timestamp(1438257913, 1),
                    "optimeDate" : ISODate("2015-07-30T12:05:13Z"),
                    "electionTime" : Timestamp(1438254975, 2),
                    "electionDate" : ISODate("2015-07-30T11:16:15Z"),
                    "configVersion" : 7,
                    "self" : true
            },
            {
                    "_id" : 1,
                    "name" : "10.235.96.12:27017",
                    "health" : 1,
                    "state" : 0,
                    "stateStr" : "STARTUP",
                    "uptime" : 62663,
                    "optime" : Timestamp(0, 0),
                    "optimeDate" : ISODate("1970-01-01T00:00:00Z"),
                    "lastHeartbeat" : ISODate("2015-07-31T04:45:56.520Z"),
                    "lastHeartbeatRecv" : ISODate("1970-01-01T00:00:00Z"),
                    "pingMs" : 0,
                    "configVersion" : -2
            },
            {
                    "_id" : 2,
                    "name" : "10.235.96.13:27017",
                    "health" : 1,
                    "state" : 0,
                    "stateStr" : "STARTUP",
                    "uptime" : 60043,
                    "lastHeartbeat" : ISODate("2015-07-31T04:45:55.786Z"),
                    "lastHeartbeatRecv" : ISODate("1970-01-01T00:00:00Z"),
                    "pingMs" : 0,
                    "configVersion" : -2
            }
    ],
    "ok" : 1
}

When I try to do a rs.conf on the Secondary or Arbiter I get the message

2015-07-31T05:23:07.927+0000 E QUERY    Error: Could not retrieve replica set config: {
    "info" : "run rs.initiate(...) if not yet done for the set",
    "ok" : 0,
    "errmsg" : "no replset config has been received",
    "code" : 94
}

I did try multiple times to clean all the local.0, local.1 files in all the machines and also removed all the journal files from the journal folder as well. Still I'm getting the same issue. Can someone tell me what is it I'm doing wrong here?

I have set replset on all the members. Tried starting in standalone mode and it works.

6

From "BOSPROD9", try to connect with the mongoshell to the other serves:

$ mongo --host 10.235.96.12 --port 27017

$ mongo --host 10.235.96.13 --port 27017

(Telnet is not the same.) If this doesn't work, it might be firewall or BindIP.

Check bind_ip (should be 0.0.0.0, change in mongodb.conf is it's 127.0.0.1):

$ netstat -nap | grep :27017 | grep LISTEN
tcp        0      0 0.0.0.0:27018           0.0.0.0:*               LISTEN      -     

Try to look at the log-files on 10.235.96.12 and 10.235.96.13, why they are stuck. Did they receive the configuration?

Try to reconfigure this way:

mongo> var cfg = {_id:"ReplicaSet1",members:[{_id:0, host:"BOSPROD9:27017"},{_id:1, host:"10.235.96.12:27017"},{_id:2, host:"10.235.96.13:27017",arbiterOnly:true}]};
mongo> rs.reconfig(cfg);

Solution:

Use hostnames instead of ip's: give all servers a hostname, update the hosts-files and use hostnames in the rs-configuration. It seems that mongodb advices not to use ip addresses, but hostnames.

  • what's the difference between rs.add() or rs.config() ? – shuboy2014 May 18 '18 at 16:58
1

I had the same issue when i cloned a VM to make the arbiter. Similar to what aldwinaldwin said, I had to change the ip address in this line in my /etc/hosts file "192.168.1.103 mongodb-arbiter.domain.local mongodb1-arbiter" to match the ip address of the server.

0

As stated in aldwinaldwin's answer, check the connectivity of all the members

if all the connectivity works well, may be you have to make a initial sync

0

In my case it was SSL issue, in mongod logs, it was ssl handshake error for heartbeat sent to primary node.

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.