0x80131904 - An existing connection was forcibly closed by the remote host

Question

Using the Microsoft.Net v4.0 SqlClient connectivity built-in to Microsoft.Net via the System.Data.SqlClient assembly, a particular machine cannot connect to a particular set of servers. When connections are attempted, the following error is thrown:

System.Data.SqlClient.SqlException (0x80131904): A connection was successfully established with the server, but then an error occurred during the login process. (provider: SSL Provider, error: 0 - An existing connection was forcibly closed by the remote host.) ---> System.ComponentModel.Win32Exception (0x80004005): An existing connection was forcibly closed by the remote host
   at System.Data.SqlClient.SqlInternalConnection.OnError(SqlException exception, Boolean breakConnection, Action`1 wrapCloseInAction)
   at System.Data.SqlClient.TdsParser.ThrowExceptionAndWarning(TdsParserStateObject stateObj, Boolean callerHasConnectionLock, Boolean asyncClose)
   at System.Data.SqlClient.TdsParserStateObject.SNIWritePacket(SNIHandle handle, SNIPacket packet, UInt32& sniError, Boolean canAccumulate, Boolean callerHasConnectionLock)
   at System.Data.SqlClient.TdsParserStateObject.WriteSni(Boolean canAccumulate)
   at System.Data.SqlClient.TdsParserStateObject.WritePacket(Byte flushMode, Boolean canAccumulate)
   at System.Data.SqlClient.TdsParser.TdsLogin(SqlLogin rec)
   at System.Data.SqlClient.SqlInternalConnectionTds.AttemptOneLogin(ServerInfo serverInfo, String newPassword, SecureString newSecurePassword, Boolean ignoreSniOpenTimeout, TimeoutTimer timeout, Boolean withFailover)
   at System.Data.SqlClient.SqlInternalConnectionTds.LoginNoFailover(ServerInfo serverInfo, String newPassword, SecureString newSecurePassword, Boolean redirectedUserInstance, SqlConnectionString connectionOptions, SqlCredential credential, TimeoutTimer timeout)
   at System.Data.SqlClient.SqlInternalConnectionTds.OpenLoginEnlist(TimeoutTimer timeout, SqlConnectionString connectionOptions, SqlCredential credential, String newPassword, SecureString newSecurePassword, Boolean redirectedUserInstance)
   at System.Data.SqlClient.SqlInternalConnectionTds..ctor(DbConnectionPoolIdentity identity, SqlConnectionString connectionOptions, SqlCredential credential, Object providerInfo,String newPassword, SecureString newSecurePassword, Boolean redirectedUserInstance, SqlConnectionString userConnectionOptions)
   at System.Data.SqlClient.SqlConnectionFactory.CreateConnection(DbConnectionOptions options, DbConnectionPoolKey poolKey, Object poolGroupProviderInfo, DbConnectionPool pool, DbConnection owningConnection, DbConnectionOptions userOptions)
   at System.Data.ProviderBase.DbConnectionFactory.CreateNonPooledConnection(DbConnection owningConnection, DbConnectionPoolGroup poolGroup, DbConnectionOptions userOptions)
   at System.Data.ProviderBase.DbConnectionFactory.TryGetConnection(DbConnection owningConnection, TaskCompletionSource`1 retry, DbConnectionOptions userOptions, DbConnectionInternal& connection)
   at System.Data.ProviderBase.DbConnectionClosed.TryOpenConnection(DbConnection outerConnection, DbConnectionFactory connectionFactory, TaskCompletionSource`1 retry, DbConnectionOptions userOptions)
   at System.Data.SqlClient.SqlConnection.TryOpen(TaskCompletionSource`1 retry)
   at System.Data.SqlClient.SqlConnection.Open()
   at ConsoleApplication1.Module1.Main()

I can connect from other machines to the same target server using the same binary without issue. I can also connect from the problematic server if I recompile the binary against Microsoft.Net v2.0.

The target server is configured to audit failed logins, however no failures are logged in the SQL Server errorlog as a result of this issue - I can cause an audit entry if I login with an invalid SQL Server login just to be certain the auditing is working.

The code I'm using to test this issue is:

Module Module1

    Sub Main()

        Dim bUseSSL As Boolean = False

        If My.Application.CommandLineArgs.Contains("/SSL") Then
            bUseSSL = True
        End If

        If bUseSSL Then
            Console.WriteLine("Encryption enabled")
        Else
            Console.WriteLine("Encryption disabled")
        End If

        Dim cb As New Data.SqlClient.SqlConnectionStringBuilder
        cb.ApplicationName = "SqlClientTest"
        cb.DataSource = "<server_name_here>"
        cb.IntegratedSecurity = True
        cb.Encrypt = bUseSSL
        'cb.TrustServerCertificate = True
        cb.NetworkLibrary = "dbmssocn"

        Try
            Using sqlconnection As New System.Data.SqlClient.SqlConnection(cb.ConnectionString)
                Try
                    sqlconnection.Open()
                    Console.WriteLine("Connected to " & cb.DataSource)
                    Dim sqlcommand As New System.Data.SqlClient.SqlCommand("SELECT @@SERVERNAME;", sqlconnection)
                    Dim sqlreader As System.Data.SqlClient.SqlDataReader = sqlcommand.ExecuteReader
                    While sqlreader.Read
                        Console.WriteLine(sqlreader.GetString(0))
                    End While
                    sqlreader.Close()
                Catch ex As Exception
                    Console.WriteLine(cb.ConnectionString)
                    Console.WriteLine("")
                    Console.WriteLine(ex.ToString)
                End Try
                Console.WriteLine("Press 'q' to quit.")
                Dim cki As ConsoleKeyInfo = Console.ReadKey
                While cki.KeyChar.ToString.ToLower <> "q"
                    cki = Console.ReadKey
                End While
            End Using
        Catch ex As Exception
            Console.WriteLine(cb.ConnectionString)
            Console.WriteLine("")
            Console.WriteLine(ex.ToString)
        End Try
    End Sub

End Module

The same failure is logged regardless of the encryption setting, and regardless of the TrustServerCertificate setting.

Something appears to be forcing the .Net SqlClient v4.0 on this machine to use SSL even if I disable it via the connection string.

Any ideas what to check?

Answer 1

Installing Microsoft .Net Framework version 4.6.2 resolves this issue in our environment, and appears to be related to the issue presented in this Microsoft Docs migration guide for .Net Framework 4.6.

All affected machines have .Net Framework Version 4.5 or 4.5.1, and have the VMware vSockets DGRAM and vSockets STREAM Base Service Providers through the vsocklib.dll that advertise non-IFS compliance.

Why this issue only appears when attempting connections to certain SQL Server 2016 instances remains an open question.