If a Parallelism Exchange Event deadlock is victim-less, is it a problem?

Question

We're seeing a lot of these Intra-Query Parallel Thread Deadlocks in our Production environment (SQL Server 2012 SP2 - yes...I know...), however when looking at the Deadlock XML that has been captured via Extended Events, the victim-list is empty.

<victim-list />

The deadlocking appears to be between 4 threads, two with the WaitType="e_waitPipeNewRow" and two with the WaitType="e_waitPipeGetRow".

 <resource-list>
  <exchangeEvent id="Pipe13904cb620" WaitType="e_waitPipeNewRow" nodeId="19">
   <owner-list>
    <owner id="process4649868" />
   </owner-list>
   <waiter-list>
    <waiter id="process40eb498" />
   </waiter-list>
  </exchangeEvent>
  <exchangeEvent id="Pipe30670d480" WaitType="e_waitPipeNewRow" nodeId="21">
   <owner-list>
    <owner id="process368ecf8" />
   </owner-list>
   <waiter-list>
    <waiter id="process46a0cf8" />
   </waiter-list>
  </exchangeEvent>
  <exchangeEvent id="Pipe13904cb4e0" WaitType="e_waitPipeGetRow" nodeId="19">
   <owner-list>
    <owner id="process40eb498" />
   </owner-list>
   <waiter-list>
    <waiter id="process368ecf8" />
   </waiter-list>
  </exchangeEvent>
  <exchangeEvent id="Pipe4a106e060" WaitType="e_waitPipeGetRow" nodeId="21">
   <owner-list>
    <owner id="process46a0cf8" />
   </owner-list>
   <waiter-list>
    <waiter id="process4649868" />
   </waiter-list>
  </exchangeEvent>
 </resource-list>

So:

1. The Victim List is empty
2. The application running the query does not error and completes the query
3. As far as we can see, there is no obvious issue, other than that the graph is captured

Therefore, is this anything to worry about other than noise?

Edit: Thanks to Paul's answer, I can see where the issue likely occurs and appears to resolve itself with the tempdb spill.

Answer 1

I wouldn't be surprised if this is the way the deadlock graph looks when an intra-query parallel deadlock is resolved by an exchange spill (so there is no victim, except performance).

You could confirm this theory by capturing exchange spills and matching them up (or not) to the deadlock.

Writing exchange buffers to tempdb to resolve a deadlock is not ideal. Look to eliminate sequences of order-preserving operations in the execution plan (e.g. order-preserving exchanges feeding a parallel merge join). Unless it's not causing a noticeable performance problem, and you have other things to worry about.

Out of interest, is this problem likely to be exacerbated by high fragmentation/outdated statistics?

Fragmentation, no. Outdated Statistics: not in any specific sense I can think of, no. Of course unrepresentative stats are rarely a good thing generally.

The fundamental issue here is that parallelism works best when there are as few dependencies between threads as possible; preserved ordering introduces rather nasty dependencies. Things can easily get gummed up, and the only way to clear the logjam is to spill a bunch of rows held at exchanges to tempdb.

---

Note

Intra-query parallel deadlocks no longer generate xml graphs when the deadlock can be resolved with an exchange spill, following a fix released for Cumulative Update 10 for SQL Server 2017 and Cumulative Update 2 for SQL Server 2016 SP2.

Answer 2

To distinguish these non-critical, "self resolving by spill" deadlocks from more important deadlocks, some searching semantics can be applied to the Xdl structure.

The following SP will not work out of the box as it depends on ufn_ExtractSubstringsByPattern() however that method can be replaced with something that returns the distinct count directly.

ALTER view [Common].[DeadLockRecentHistoryView]
as
/*---------------------------------------------------------------------------------------------------------------------
    Purpose:  List history of recent deadlock events

    Warning:  The XML processing may hit a recursion limit (100), suggest using "option (maxrecursion 10000)".

    Xdl File:
        The SSMS deadlock file format .XDL format (xml) has changed with later versions of SQL Server.  This version tested with 2012.

    Ring Buffer issues:
        https://connect.microsoft.com/SQLServer/feedback/details/754115/xevents-system-health-does-not-catch-all-deadlocks
        https://www.sqlskills.com/blogs/jonathan/why-i-hate-the-ring_buffer-target-in-extended-events/

    Links:
        http://www.sqlskills.com/blogs/jonathan/multi-victim-deadlocks/
        https://www.sqlskills.com/blogs/jonathan/graphically-viewing-extended-events-deadlock-graphs/
        http://www.mssqltips.com/sqlservertip/1234/capturing-sql-server-deadlock-information-in-xml-format/
        http://blogs.msdn.com/b/sqldatabasetalk/archive/2013/05/01/tracking-down-deadlocks-in-sql-database.aspx
        http://dba.stackexchange.com/questions/10644/deadlock-error-isnt-returning-the-deadlock-sql/10646#10646        

    Modified    By           Description
    ----------  -----------  ------------------------------------------------------------------------------------------
    2014.10.29  crokusek     From Internet, http://stackoverflow.com/questions/19817951
    2015.05.05  crokusek     Improve so that the output is consumable by SSMS 2012 as "Open .xdl file"                             
    2015.05.22  crokusek     Remove special character for the cast to Xml (like '&')
    2017.08.03  crokusek     Abandon ring-buffer approach and use event log files.  Filter out internal deadlocks.
    2018.07.16  crokusek     Added field(s) like ProbablyHandledBySpill to help identify non-critical deadlocks.
  ---------------------------------------------------------------------------------------------------------------------*/
with XmlDeadlockReports as
(
  select convert(xml, event_data) as EventData         
    from sys.fn_xe_file_target_read_file(N'system_health*.xel', NULL, NULL, NULL)      
   where substring(event_data, 1, 50) like '%"xml_deadlock_report"%'       
)
select top 10000
       EventData.value('(event/@timestamp)[1]', 'datetime2(7)') as CreatedUtc,
       --(select TimePst from Common.ufn_ConvertUtcToPst(EventData.value('(event/@timestamp)[1]', 'datetime2(7)'))) as CreatedPst,
       DistinctSpidCount,       
       HasExchangeEvent,
       IsVictimless,                  
       --
       -- If the deadlock contains Exchange Events and lists no victims, it probably occurred
       -- during execution of a single query that contained parallellism but got stuck due to 
       -- ordering issues.   https://dba.stackexchange.com/questions/197779
       -- 
       -- These will not raise an exception to the caller and will complete by spilling to tempdb
       -- however they may run much slower than they would without the spill(s).
       --
       convert(bit, iif(DistinctSpidCount = 1 and HasExchangeEvent = 1 and IsVictimless = 1, 1, 0)) as ProbablyHandledBySpill,
       len(et.XdlFileText) as LenXdlFile,
       eddl.XdlFile as XdlFile
  from XmlDeadlockReports
 cross apply 
     ( 
       select eventData.query('event/data/value/deadlock') as XdlFile 
     ) eddl
 cross apply 
     ( 
        select convert(nvarchar(max), eddl.XdlFile) as XdlFileText 
     ) as et
 cross apply 
     (
       select count(distinct Match) as DistinctSpidCount
         from common.ufn_ExtractSubstringsByPattern(et.XdlFileText, 'spid="%%"')
     ) spids
 cross apply
     (
       select convert(bit, iif(charindex('<exchangeEvent', et.XdlFileText) > 0, 1, 0)) as HasExchangeEvent,
              --
              convert(bit, iif(     charindex('<victim-list>', et.XdlFileText) = 0
                                and charindex('<victim-list/>', et.XdlFileText) > 0, 1, 0)) as IsVictimless
     ) as flags        
 order by CreatedUtc desc
GO