Unable to Force a WSFC Cluster to Start Without a Quorum

Question

I was testing a three node AlwaysOn availability group cluster. SQL Server is on Node 1 and I powered off Node 1 and Node 3 simultaneously. The cluster lost its quorum and appear to be down. I tried following "Force a WSFC Cluster to Start Without a Quorum", but I am unable to bring the cluster online. It is stuck is "joining" state.

What can we do now?

cluster log

Initially the cluster service is down. After executing the Start-ClusterNode command it goes into running state. I have tried restarting the cluster service and also restarting the node, but can't force quorum.

Answer 1

In the screenshot it shows the attempt to use -FixQuorum as part of the Start-ClusterNode cluster powershell commands, however the cluster log shows that quorum was not forced.

Your log shows the following:

Creating cluster node <vector len='1'>
<item>ClusSvc</item>
</vector>

Which is just starting up the cluster service. The log is also missing the fact stating that on this node quorum was forced.

Here is an example of how a cluster log (WSFC 2016, like the one you posted) would look if you forced quorum on the node:

Creating cluster node <vector len='2'>
<item>ClusSvc</item>
<item>/FixQuorum</item>
</vector>  

Additionally, you'd see another entry further down:

[CS] FixQuorum mode is on

My suggestion is to log locally onto the server and try running the powershell command (after shutting down the cluster service if it's still running):

Start-ClusterNode -Node SQL2 -ForceQuorum

Then check the status about 15 seconds later with:

Get-ClusterNode -Node SQL2

Check to see what is the status and also check the cluster log to see if the ForceQuorum switch was properly passed.