We are currently testing a SQL Server 2012 AlwaysOn HA implementation where we have 2 out of the 3 nodes are synchronous. The third, asynchronous node, does not have a vote in the quorum. There is a scenario in which the first two nodes experience complete failure. When this happens, the third node is up and running but it has it's database in the AG marked as "Recovery Pending". How can I get this database out of this state and make it available?


The first attempt would be to try to get either the primary replica back online, or to get the synchronized secondary replica online to ensure that there is no data loss. But provided this is a true disaster recovery scenario (as your original question implies), then it would seem as though your actual cluster (WSFC) has failed. With the database entering the recovery_pending state, you can't unjoin the database to recover it. Likewise, you would need the cluster in order to do a failover. So the following assumes that you can't force quorum on either of the sync replicas, and you need to failover to the async.

The next step at this point to recover to the async secondary that is currently online, would be to force quorum (WSFC) and then perform a forced failover (with the potential for data loss).

For more information, I highly recommend you read up on BOL regarding the topic Perform a Forced Manual Failover of an Availability Group

  • Thank you for pointing me in the right direction. Unfortunately I am unable to get the cluster to start on the 3rd node. According to this msdn article, I should be able to launch Cluster Manager and select Force Cluster Start in the actions pane. msdn.microsoft.com/en-us/library/hh270275(v=SQL.110).aspx However, there is no Force Cluster Start option in the pane. It also looks like I'm not connected to the cluster and when I try to, it states the cluster is in the process of being started. – Geoff Dawdy Oct 29 '14 at 15:15
  • Have you tried any way besides fc admin? – Thomas Stringer Oct 29 '14 at 20:51

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.