we are running a mongo 2.6.5, 5 member replica set on CentOS and are looking to upgrade to 4.3.3. We do not use sharding. The tricky part for us is that we operate a service and can't have scheduled maintenance windows of very big size, so our hope is to minimize downtime to a few 10s of seconds early some morning. Our hope is that we could upgrade the cluster by just adding a new node to the cluster - one that is running newer mongo - let it get all sync'd up, and then remove one of the nodes running 2.6, and repeating the process until all the nodes are up to date.
So rather than actually upgrading nodes, we'd really be doing a fresh install on each one - we don't have to run a node's copy of the data through any sort of upgrade process but instead will treat it like temporarily expanding the replica set's size with a new server. Our desire to do it this way is to minimize downtime but also to allow us to proceed slowly - we want to bring up a new node and just let the dust settle for awhile, run some tests, etc. before proceeding. There are different elements of risk no matter which way we go, but our preference is for a process that changes few things at a time even if the overall end-to-end time is longer (we like lots of incremental changes spread out over time).
But when I try to add 4.3.3 nodes to 2.6.5 cluster, I get the following error:
"stateStr" : "(not reachable/healthy)",
When I take a look at the logs, it says:
2022-06-01T19:13:26.080+0000 [rsHealthPoll] couldn't connect to node1:27017: couldn't connect to server node1:27017 failed, connection attempt failed 2022-06-01T19:13:26.137+0000 [rsHealthPoll] warning: Failed to connect to 10.x.x.x:27017, reason: errno:111 Connection refused
Can you please help me?