Have a couple Fiber channel SAN-attached cluster servers which have been intermittently reporting errors like:
2012-05-09 16:31:54.13 spid74 Error: 823, Severity: 24, State: 2. 2012-05-09 16:31:54.13 spid74 The operating system returned error 38(Reached the end of the file.) to SQL Server during a read at offset 0x000000e11b0000 in file 'T:\tempdb.mdf'. Additional messages in the SQL Server error log and system event log may provide more detail. This is a severe system-level error condition that threatens database integrity and must be corrected immediately. Complete a full database consistency check (DBCC CHECKDB). This error can be caused by many factors; for more information, see SQL Server Books Online. 2012-05-09 16:31:54.13 spid74 Error while allocating extent for a worktable. Extent (1:461024) in TEMPDB may have been lost.
SQL Server re-start, which re-creates tempdb, has so far always fixed the issue.
Per a Microsoft tech recommendation, I've enabled trace flag 818 ( http://support.microsoft.com/kb/826433 ), but this flag has not yet yielded any fruit
[addition ERRORLOG details]; I'm supposing what's happening must not be meeting the criteria for this flag's feature to log extra data.
Have a Microsoft case open, and sysadmins and storage vendor looking at things. Also working through http://support.microsoft.com/kb/2015755 , including running sqliosim at one site for ~2.5 hrs. Though there must be something going wrong in the I/O path, nothing has bubbled to the top to pinpoint the cause.
- What have you experienced?
- What are your thoughts about where else to look?