1

We have an application in Azure which connects to SQL Server on-premise over hybrid connection
Every 4-5 requests from 1000 would fail with error 10054:

A transport-level error has occurred when receiving results from the server.
An existing connection was forcibly closed by the remote host

What we've tried:

  • restarted app in Azure
  • increased ConnectTimeOut and CommandTimeOut in application
  • rebooted Windows hosting SQL Server
  • installed OS and SQL Server patches on-prem
  • checked SQL Server error log, no related errors
  • sysadmin checked firewall/switches - ZERO packet loss or drops
  • performed update of firmware and drivers (network, sas, raid)

Also checked this:

  • remote access in SQL Server is turned on, obviously
  • SQL Server Browser turned off, but there is only one default instance
  • hybrid connection manager is up to date
  • SqlClient version is up to date (System.Data.SqlClient: 4.0.0.0)
  • databases auto_close = OFF
  • sessions are not getting killed by anyone
  • server does not go to sleep or energy saving mode
  • TCP chimney is disabled
  • SPN config is OK with Kerberos Configuration Manager

None of above helped. Resolution:

https://learn.microsoft.com/en-us/previous-versions/sql/sql-server-2008-r2/ms187005(v=sql.105)?redirectedfrom=MSDN

Although link is for Windows Server 2003, it worked for Windows Server 2016
Solution was to add registry entry SynAttackProtect - DWORD - 00000000

HKEY_LOCAL_MACHINE\SYSTEM\CurrentControlSet\Services\Tcpip\Parameters\

As we understand, requests from Azure were too rapid sometimes so system thought it was under attack ? We added registry entry and stopped getting error 10054. Team is happy and already moved on to other tasks

I thought posting above information on DBA SE may help other guys troubleshooting this issue

Question:

Is this solution optimal, as it seem to open server to SYN-flood and denial-of-service attacks ?
Are there any alternatives in fixing 10054 error ?

Improve this question
3
  • Which client driver are you using?
    – Charlieface
    Jun 28, 2022 at 20:15
  • @Charlieface this is what I've got from developers: App is using Dapper Micro ORM over native SQL libraries to connect to the database: Dapper versions: 1.42.0 & 1.50.5 System.Data.SqlClient: 4.0.0.0
    – Aleksey Vitsko
    Jul 6, 2022 at 14:25
  • 1
    How many clients do you have making requests concurrently? Connection pooling should normally keep the number of physical connections down to 1 or 2 per client. I suspect connection pooling is not working properly, probably because the application is not disposing/closing the SqlConnection object correctly. So every time the client makes a new request then a new physical connection is opened, requiring a TCP SYN/ACK handshake, as well as TLS and TDS handshakes, all of which could cause you problems if done too many times.
    – Charlieface
    Jul 6, 2022 at 14:29

0

Reset to default

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge that you have read and understand our privacy policy and code of conduct.

Browse other questions tagged or ask your own question.