OLE DB provider "SQLNCLI11" for linked server "001" returned message "Unable to complete login process due to delay in opening server connection". Msg 7303, Level 16, State 1, Line 19 Cannot initialize the data source object of OLE DB provider "SQLNCLI11" for linked server "001".
I realize there are a lot of potentially applicable SO posts and google hits on this (many of them I've read quite a few times), so let me go through what we're seeing and what we've already tried. I'm at the point where I'm honestly not sure what else to look at.
We have Linked Server 001 (Dest) going to 002 (Source) as a SQL Login which is called from C# / .Net application code as a proc via a data access method and an SSPI connection string via service account with proper rights. I believe the .Net version was 4.3 and is now 4.7, but the issue was persistent in both. It's been migrated from an older 2008R2 environment on a different network where this was never an issue on physical boxes and both of the new, current are 64-bit 2016 virtuals.
It is consistently reproducible during the issues, but not otherwise and seems to always be the same call. Nothing shows up in sp_what for blocking/deadlocks/weird locked pages or objects. And the issue isn't reproducible when it's not being reported. Sometimes it only happens once or twice, sometimes for 10-15 and then it'll work fine for weeks. Not sure it's related, but the application also shows thread aborts occasionally when we see this, but I think it's more of a symptom.
- The linked server and account on both sides have been recreated manually and via script as well as GUI and from known working linked servers. I've verified that they're not getting locked out and have the correct permissions multiple times.
- I've increased the max query timeout on both sides.
- The SQLNCLI11.DLL drivers are on both servers in the same versioning as the SQL Server install.
- Nothing obvious is showing up in the SQL Management or Error logs in terms of login failures, corruption, or weird behavior.
- We don't see anything in the server event logs on either side.
- Our networking and security teams have taken a look and aren't finding any blips in connectivity and traffic the boxes reside on the same switch and don't pass through a firewall.
- Ports are open.
- When these issues are occurring Test Connection from the linked server works fine and I can typically get to the underlying issue table via ad-hoc, but not the stored proc's code, even run in-place.
What am I missing here?