We're getting ready to perform a large upgrade on our SQL Servers and are noticing some unusual behavior with Distributed Availability Groups that I'm trying to resolve before moving forward.
Last month, I upgraded a remote secondary server from SQL Server 2016 to SQL Server 2017. This server is a part of multiple Distributed Availability Groups (DAGs) and a separate Availability Group (AG). When we upgraded this server, we were unaware that it would get into an unreadable state, so during the past month we've solely been relying on the primary server.
As a part of the upcoming upgrade, I applied the CU 4 patch to the server and rebooted it. When the server came back online, the just-patched secondary showed all of the DAGs/AGs were syncing without any issues.
However, the primary was showing a very different story. It was reporting that
- the separate AG was syncing without any issues
- but the DAGs were in a Not Synchronzing / Not Healthy state
After initially panicking, I attempted the following things to get things synchronizing again in the DAGs:
- From the primary, I stopped and resumed the data movement. This did not start syncing the data.
- On the secondary (the one I just patched) I ran
ALTER DATABASE [<database] SET HADR RESUME;- which execute without errors, but did not resume any syncing
My last attempt at syncing the data again was to login to the secondary, and manually restart the SQL Server service. Manually restarting the service seems a bit extreme, as I'd expect the server being rebooted would have been enough.
Has anyone run into this issue where a DAG doesn't start syncing to a secondary after a reboot? If so, how was it resolved?
I checked both the SQL Server error log, and the event viewer on the secondary server, there was nothing out of the ordinary that I could see.