We have an application in Azure which connects to SQL Server on-premise over hybrid connection
Every 4-5 requests from 1000 would fail with error 10054:
A transport-level error has occurred when receiving results from the server.
An existing connection was forcibly closed by the remote host
What we've tried:
- restarted app in Azure
- increased ConnectTimeOut and CommandTimeOut in application
- rebooted Windows hosting SQL Server
- installed OS and SQL Server patches on-prem
- checked SQL Server error log, no related errors
- sysadmin checked firewall/switches - ZERO packet loss or drops
- performed update of firmware and drivers (network, sas, raid)
Also checked this:
- remote access in SQL Server is turned on, obviously
- SQL Server Browser turned off, but there is only one default instance
- hybrid connection manager is up to date
- SqlClient version is up to date (System.Data.SqlClient: 126.96.36.199)
- databases auto_close = OFF
- sessions are not getting killed by anyone
- server does not go to sleep or energy saving mode
- TCP chimney is disabled
- SPN config is OK with Kerberos Configuration Manager
None of above helped. Resolution:
Although link is for Windows Server 2003, it worked for Windows Server 2016
Solution was to add registry entry SynAttackProtect - DWORD - 00000000
As we understand, requests from Azure were too rapid sometimes so system thought it was under attack ? We added registry entry and stopped getting error 10054. Team is happy and already moved on to other tasks
I thought posting above information on DBA SE may help other guys troubleshooting this issue
Is this solution optimal, as it seem to open server to SYN-flood and denial-of-service attacks ?
Are there any alternatives in fixing 10054 error ?
SqlConnectionobject correctly. So every time the client makes a new request then a new physical connection is opened, requiring a TCP SYN/ACK handshake, as well as TLS and TDS handshakes, all of which could cause you problems if done too many times.