We are testing our SQL Server database for concurrency issues and it appears that a single, simple update statement being run concurrently in transactions is resulting in deadlocks.
Here is the statement:
(@Version nvarchar(7),@TestAttemptId uniqueidentifier,@TestFormName nvarchar(1),
@ActorId uniqueidentifier,@TimeStamp datetime,@Id uniqueidentifier,@UpdatedOn
datetime,@UpdatedBy uniqueidentifier)
UPDATE [dbo].[TestAttempts]
Set [TestFormName]=@TestFormName,[UpdatedOn]=@UpdatedOn,
[UpdatedBy]=@UpdatedBy
Where [Id]=@Id
Here's the diagram provided by the xdl we've retrieved for one of the deadlocks:
Since the primary key is one of the problem locks, here's the primary key's definition:
ALTER TABLE [dbo].[TestAttempts] ADD CONSTRAINT [PK_TestAttempts] PRIMARY KEY CLUSTERED
(
[Id] ASC,
[TestAdministrationId] ASC,
[TestTakerId] ASC
)WITH (PAD_INDEX = OFF, STATISTICS_NORECOMPUTE = OFF, SORT_IN_TEMPDB = OFF, IGNORE_DUP_KEY = OFF, ONLINE = OFF, ALLOW_ROW_LOCKS = ON, ALLOW_PAGE_LOCKS = ON, OPTIMIZE_FOR_SEQUENTIAL_KEY = OFF) ON [PRIMARY]
GO
This concurrency test essentially involves the same code being run multiple times concurrently but updating different records. I'm fairly certain that there aren't any queries being run in the transaction prior to this update that would lock the key or the table. It's mainly a sequence of updates being performed, and this particular update only runs once in each process. So I'm confused as to how the same update query could deadlock here based only its primary key and the table itself. I'd expect each individual process to access the objects in the same sequence.
--- EDIT ---
As requested, here's the full DDL for the table and its indexes:
Create Table [dbo].[TestAttempts]
(
[Id] UniqueIdentifier Not Null,
[TestTakerId] Int Not Null,
[TestAdministrationId] UniqueIdentifier Not Null,
[TestFormName] NVARCHAR(50) NULL,
[DocumentId] UniqueIdentifier Null,
[TimeRemaining] TIME NULL,
[Score] Decimal(18, 5) Null,
[Pass] Bit Null,
[StartedOn] DateTime Null,
[EndedOn] DateTime Null,
[LocationName] NVARCHAR(255) NULL,
[KioskUsed] BIT NOT NULL DEFAULT 0,
[STAUsed] BIT NOT NULL DEFAULT 0,
[CreatedOn] DateTime Not Null,
[CreatedBy] UniqueIdentifier Not Null,
[UpdatedOn] DateTime Null,
[UpdatedBy] UniqueIdentifier Null,
[Notes] NTEXT NULL,
[WhiteboardHistory] NTEXT NULL,
Constraint [FK_TestAttempts_Documents] Foreign Key ([DocumentId]) References [dbo].[Documents]([Id]),
Constraint [PK_TestAttempts] Primary Key Clustered ([Id], [TestAdministrationId], [TestTakerId])
) On [PRIMARY]
Go
Create Index [IX_TestAttempts_TestAdministrationId_TestTakerId] On [dbo].[TestAttempts]([TestAdministrationId], [TestTakerId])
Go
Create Index [IX_TestAttempts_DocumentId] On [dbo].[TestAttempts]([DocumentId])
GO
CREATE NONCLUSTERED INDEX [_dta_index_TestAttempts_17_591456956__K2_K3_K1_K10] ON [dbo].[TestAttempts]
(
[TestTakerId] ASC,
[TestAdministrationId] ASC,
[Id] ASC,
[EndedOn] ASC
)WITH (SORT_IN_TEMPDB = OFF, DROP_EXISTING = OFF, ONLINE = OFF) ON [PRIMARY]
GO
CREATE NONCLUSTERED INDEX [_dta_index_TestAttempts_17_591456956__K1_K3_K4] ON [dbo].[TestAttempts]
(
[Id] ASC,
[TestAdministrationId] ASC,
[TestFormName] ASC
)WITH (SORT_IN_TEMPDB = OFF, DROP_EXISTING = OFF, ONLINE = OFF) ON [PRIMARY]
GO
CREATE NONCLUSTERED INDEX [_dta_index_TestAttempts_17_591456956__K10_K3_K2_K1_K9] ON [dbo].[TestAttempts]
(
[EndedOn] ASC,
[TestAdministrationId] ASC,
[TestTakerId] ASC,
[Id] ASC,
[StartedOn] ASC
)WITH (SORT_IN_TEMPDB = OFF, DROP_EXISTING = OFF, ONLINE = OFF) ON [PRIMARY]
And the full XDL deadlock graph:
<deadlock>
<victim-list>
<victimProcess id="process239bfc75088" />
</victim-list>
<process-list>
<process id="process239bfc75088" taskpriority="0" logused="31516" waitresource="KEY: 8:72057594060865536 (3f537378e06c)" waittime="177" ownerId="785882138" transactionname="user_transaction" lasttranstarted="2023-08-29T18:24:57.497" XDES="0x320af08c470" lockMode="U" schedulerid="1" kpid="15476" status="suspended" spid="203" sbid="2" ecid="0" priority="0" trancount="2" lastbatchstarted="2023-08-29T18:24:58.853" lastbatchcompleted="2023-08-29T18:24:58.823" lastattention="1900-01-01T00:00:00.823" clientapp="Core .Net SqlClient Data Provider" hostname="waiter" hostpid="5800" loginname="XXXXXX" isolationlevel="read committed (2)" xactid="785882138" currentdb="8" currentdbname="Beta" lockTimeout="4294967295" clientoption1="671088672" clientoption2="128056">
<stackFrames>
<frame id="00" address="0x7FFCAB9600E4" pdb="ntdll.pdb" age="1" guid="5F27FA82-A8B9-9D6A-5D89-9A321CF31DDB" module="ntdll" rva="0xA00E4" />
<frame id="01" address="0x7FFCA923D77E" pdb="kernelbase.pdb" age="1" guid="DD9BA441-0270-5FC1-972E-B050EB24E81E" module="kernelbase" rva="0x3D77E" />
<frame id="02" address="0x7FFC86CA1F52" pdb="SqlDK.pdb" age="2" guid="EE37F036-82E0-483A-9A70-6A5D6C23610C" module="SqlDK" rva="0x1F52" />
<frame id="03" address="0x7FFC86CA1C6B" pdb="SqlDK.pdb" age="2" guid="EE37F036-82E0-483A-9A70-6A5D6C23610C" module="SqlDK" rva="0x1C6B" />
<frame id="04" address="0x7FFC86CA38D7" pdb="SqlDK.pdb" age="2" guid="EE37F036-82E0-483A-9A70-6A5D6C23610C" module="SqlDK" rva="0x38D7" />
<frame id="05" address="0x7FFC87337445" pdb="sqlmin.pdb" age="2" guid="338E8B8A-88B6-45EF-BDDB-D121DEF6A398" module="sqlmin" rva="0x87445" />
<frame id="06" address="0x7FFC87337243" pdb="sqlmin.pdb" age="2" guid="338E8B8A-88B6-45EF-BDDB-D121DEF6A398" module="sqlmin" rva="0x87243" />
<frame id="07" address="0x7FFC872C0D94" pdb="sqlmin.pdb" age="2" guid="338E8B8A-88B6-45EF-BDDB-D121DEF6A398" module="sqlmin" rva="0x10D94" />
<frame id="08" address="0x7FFC872C9B58" pdb="sqlmin.pdb" age="2" guid="338E8B8A-88B6-45EF-BDDB-D121DEF6A398" module="sqlmin" rva="0x19B58" />
<frame id="09" address="0x7FFC8824CD21" pdb="sqlmin.pdb" age="2" guid="338E8B8A-88B6-45EF-BDDB-D121DEF6A398" module="sqlmin" rva="0xF9CD21" />
<frame id="10" address="0x7FFC872C9E72" pdb="sqlmin.pdb" age="2" guid="338E8B8A-88B6-45EF-BDDB-D121DEF6A398" module="sqlmin" rva="0x19E72" />
<frame id="11" address="0x7FFC872C0550" pdb="sqlmin.pdb" age="2" guid="338E8B8A-88B6-45EF-BDDB-D121DEF6A398" module="sqlmin" rva="0x10550" />
<frame id="12" address="0x7FFC872D955B" pdb="sqlmin.pdb" age="2" guid="338E8B8A-88B6-45EF-BDDB-D121DEF6A398" module="sqlmin" rva="0x2955B" />
<frame id="13" address="0x7FFC872D9476" pdb="sqlmin.pdb" age="2" guid="338E8B8A-88B6-45EF-BDDB-D121DEF6A398" module="sqlmin" rva="0x29476" />
<frame id="14" address="0x7FFC8749724F" pdb="sqlmin.pdb" age="2" guid="338E8B8A-88B6-45EF-BDDB-D121DEF6A398" module="sqlmin" rva="0x1E724F" />
<frame id="15" address="0x7FFC872D95D9" pdb="sqlmin.pdb" age="2" guid="338E8B8A-88B6-45EF-BDDB-D121DEF6A398" module="sqlmin" rva="0x295D9" />
<frame id="16" address="0x7FFC87314495" pdb="sqlmin.pdb" age="2" guid="338E8B8A-88B6-45EF-BDDB-D121DEF6A398" module="sqlmin" rva="0x64495" />
<frame id="17" address="0x7FFC872D95D9" pdb="sqlmin.pdb" age="2" guid="338E8B8A-88B6-45EF-BDDB-D121DEF6A398" module="sqlmin" rva="0x295D9" />
<frame id="18" address="0x7FFC873141CB" pdb="sqlmin.pdb" age="2" guid="338E8B8A-88B6-45EF-BDDB-D121DEF6A398" module="sqlmin" rva="0x641CB" />
<frame id="19" address="0x7FFC872D95D9" pdb="sqlmin.pdb" age="2" guid="338E8B8A-88B6-45EF-BDDB-D121DEF6A398" module="sqlmin" rva="0x295D9" />
<frame id="20" address="0x7FFC872F59CF" pdb="sqlmin.pdb" age="2" guid="338E8B8A-88B6-45EF-BDDB-D121DEF6A398" module="sqlmin" rva="0x459CF" />
<frame id="21" address="0x7FFC83E30BC8" pdb="sqllang.pdb" age="2" guid="C58AAD30-E220-4D32-A11E-C03D11F62263" module="sqllang" rva="0x10BC8" />
<frame id="22" address="0x7FFC83E4F67C" pdb="sqllang.pdb" age="2" guid="C58AAD30-E220-4D32-A11E-C03D11F62263" module="sqllang" rva="0x2F67C" />
<frame id="23" address="0x7FFC83E4F98A" pdb="sqllang.pdb" age="2" guid="C58AAD30-E220-4D32-A11E-C03D11F62263" module="sqllang" rva="0x2F98A" />
<frame id="24" address="0x7FFC83E2D62F" pdb="sqllang.pdb" age="2" guid="C58AAD30-E220-4D32-A11E-C03D11F62263" module="sqllang" rva="0xD62F" />
<frame id="25" address="0x7FFC83E2DA7C" pdb="sqllang.pdb" age="2" guid="C58AAD30-E220-4D32-A11E-C03D11F62263" module="sqllang" rva="0xDA7C" />
<frame id="26" address="0x7FFC83E2EF18" pdb="sqllang.pdb" age="2" guid="C58AAD30-E220-4D32-A11E-C03D11F62263" module="sqllang" rva="0xEF18" />
<frame id="27" address="0x7FFC83E2E5D1" pdb="sqllang.pdb" age="2" guid="C58AAD30-E220-4D32-A11E-C03D11F62263" module="sqllang" rva="0xE5D1" />
<frame id="28" address="0x7FFC84C4D72E" pdb="sqllang.pdb" age="2" guid="C58AAD30-E220-4D32-A11E-C03D11F62263" module="sqllang" rva="0xE2D72E" />
<frame id="29" address="0x7FFC84C751EF" pdb="sqllang.pdb" age="2" guid="C58AAD30-E220-4D32-A11E-C03D11F62263" module="sqllang" rva="0xE551EF" />
<frame id="30" address="0x7FFC84C75A4A" pdb="sqllang.pdb" age="2" guid="C58AAD30-E220-4D32-A11E-C03D11F62263" module="sqllang" rva="0xE55A4A" />
<frame id="31" address="0x7FFC83E38243" pdb="sqllang.pdb" age="2" guid="C58AAD30-E220-4D32-A11E-C03D11F62263" module="sqllang" rva="0x18243" />
<frame id="32" address="0x600000001" />
<frame id="33" address="0x180AC9B9338C" />
</stackFrames>
<executionStack>
<frame procname="adhoc" line="1" stmtstart="390" stmtend="632" sqlhandle="0x020000004415d703bb1554faf0fea77efb23bd399a83affe0000000000000000000000000000000000000000">
unknown </frame>
<frame procname="unknown" line="1" sqlhandle="0x0000000000000000000000000000000000000000000000000000000000000000000000000000000000000000">
unknown </frame>
</executionStack>
<inputbuf>
(@Version nvarchar(7),@TestAttemptId uniqueidentifier,@TestFormName nvarchar(1),@ActorId uniqueidentifier,@TimeStamp datetime,@Id uniqueidentifier,@UpdatedOn datetime,@UpdatedBy uniqueidentifier)UPDATE [dbo].[TestAttempts] Set [TestFormName]=@TestFormName,[UpdatedOn]=@UpdatedOn,[UpdatedBy]=@UpdatedBy Where [Id]=@Id </inputbuf>
</process>
<process id="process2fa8801fc28" taskpriority="0" logused="32296" waitresource="KEY: 8:72057594060865536 (b6b85bb220f6)" waittime="179" ownerId="785882150" transactionname="user_transaction" lasttranstarted="2023-08-29T18:24:57.560" XDES="0x267fc1f8470" lockMode="U" schedulerid="8" kpid="13436" status="suspended" spid="151" sbid="2" ecid="0" priority="0" trancount="2" lastbatchstarted="2023-08-29T18:24:58.903" lastbatchcompleted="2023-08-29T18:24:58.887" lastattention="1900-01-01T00:00:00.887" clientapp="Core .Net SqlClient Data Provider" hostname="waiter" hostpid="5800" loginname="XXXXXXXXXXXX" isolationlevel="read committed (2)" xactid="785882150" currentdb="8" currentdbname="Beta" lockTimeout="4294967295" clientoption1="671088672" clientoption2="128056">
<stackFrames>
<frame id="00" address="0x7FFCAB963814" pdb="ntdll.pdb" age="1" guid="5F27FA82-A8B9-9D6A-5D89-9A321CF31DDB" module="ntdll" rva="0xA3814" />
<frame id="01" address="0x7FFCA92C69B9" pdb="kernelbase.pdb" age="1" guid="DD9BA441-0270-5FC1-972E-B050EB24E81E" module="kernelbase" rva="0xC69B9" />
<frame id="02" address="0x7FFC86CA9CF7" pdb="SqlDK.pdb" age="2" guid="EE37F036-82E0-483A-9A70-6A5D6C23610C" module="SqlDK" rva="0x9CF7" />
<frame id="03" address="0x7FFC86CA9C12" pdb="SqlDK.pdb" age="2" guid="EE37F036-82E0-483A-9A70-6A5D6C23610C" module="SqlDK" rva="0x9C12" />
<frame id="04" address="0x7FFC86CA1C6B" pdb="SqlDK.pdb" age="2" guid="EE37F036-82E0-483A-9A70-6A5D6C23610C" module="SqlDK" rva="0x1C6B" />
<frame id="05" address="0x7FFC86CA38D7" pdb="SqlDK.pdb" age="2" guid="EE37F036-82E0-483A-9A70-6A5D6C23610C" module="SqlDK" rva="0x38D7" />
<frame id="06" address="0x7FFC87337445" pdb="sqlmin.pdb" age="2" guid="338E8B8A-88B6-45EF-BDDB-D121DEF6A398" module="sqlmin" rva="0x87445" />
<frame id="07" address="0x7FFC87337243" pdb="sqlmin.pdb" age="2" guid="338E8B8A-88B6-45EF-BDDB-D121DEF6A398" module="sqlmin" rva="0x87243" />
<frame id="08" address="0x7FFC872C0D94" pdb="sqlmin.pdb" age="2" guid="338E8B8A-88B6-45EF-BDDB-D121DEF6A398" module="sqlmin" rva="0x10D94" />
<frame id="09" address="0x7FFC872C9B58" pdb="sqlmin.pdb" age="2" guid="338E8B8A-88B6-45EF-BDDB-D121DEF6A398" module="sqlmin" rva="0x19B58" />
<frame id="10" address="0x7FFC8824CD21" pdb="sqlmin.pdb" age="2" guid="338E8B8A-88B6-45EF-BDDB-D121DEF6A398" module="sqlmin" rva="0xF9CD21" />
<frame id="11" address="0x7FFC872C9E72" pdb="sqlmin.pdb" age="2" guid="338E8B8A-88B6-45EF-BDDB-D121DEF6A398" module="sqlmin" rva="0x19E72" />
<frame id="12" address="0x7FFC872C0550" pdb="sqlmin.pdb" age="2" guid="338E8B8A-88B6-45EF-BDDB-D121DEF6A398" module="sqlmin" rva="0x10550" />
<frame id="13" address="0x7FFC872D955B" pdb="sqlmin.pdb" age="2" guid="338E8B8A-88B6-45EF-BDDB-D121DEF6A398" module="sqlmin" rva="0x2955B" />
<frame id="14" address="0x7FFC872D9476" pdb="sqlmin.pdb" age="2" guid="338E8B8A-88B6-45EF-BDDB-D121DEF6A398" module="sqlmin" rva="0x29476" />
<frame id="15" address="0x7FFC8749724F" pdb="sqlmin.pdb" age="2" guid="338E8B8A-88B6-45EF-BDDB-D121DEF6A398" module="sqlmin" rva="0x1E724F" />
<frame id="16" address="0x7FFC872D95D9" pdb="sqlmin.pdb" age="2" guid="338E8B8A-88B6-45EF-BDDB-D121DEF6A398" module="sqlmin" rva="0x295D9" />
<frame id="17" address="0x7FFC87314495" pdb="sqlmin.pdb" age="2" guid="338E8B8A-88B6-45EF-BDDB-D121DEF6A398" module="sqlmin" rva="0x64495" />
<frame id="18" address="0x7FFC872D95D9" pdb="sqlmin.pdb" age="2" guid="338E8B8A-88B6-45EF-BDDB-D121DEF6A398" module="sqlmin" rva="0x295D9" />
<frame id="19" address="0x7FFC873141CB" pdb="sqlmin.pdb" age="2" guid="338E8B8A-88B6-45EF-BDDB-D121DEF6A398" module="sqlmin" rva="0x641CB" />
<frame id="20" address="0x7FFC872D95D9" pdb="sqlmin.pdb" age="2" guid="338E8B8A-88B6-45EF-BDDB-D121DEF6A398" module="sqlmin" rva="0x295D9" />
<frame id="21" address="0x7FFC872F59CF" pdb="sqlmin.pdb" age="2" guid="338E8B8A-88B6-45EF-BDDB-D121DEF6A398" module="sqlmin" rva="0x459CF" />
<frame id="22" address="0x7FFC83E30C8D" pdb="sqllang.pdb" age="2" guid="C58AAD30-E220-4D32-A11E-C03D11F62263" module="sqllang" rva="0x10C8D" />
<frame id="23" address="0x7FFC83E4F67C" pdb="sqllang.pdb" age="2" guid="C58AAD30-E220-4D32-A11E-C03D11F62263" module="sqllang" rva="0x2F67C" />
<frame id="24" address="0x7FFC83E4F98A" pdb="sqllang.pdb" age="2" guid="C58AAD30-E220-4D32-A11E-C03D11F62263" module="sqllang" rva="0x2F98A" />
<frame id="25" address="0x7FFC83E2D62F" pdb="sqllang.pdb" age="2" guid="C58AAD30-E220-4D32-A11E-C03D11F62263" module="sqllang" rva="0xD62F" />
<frame id="26" address="0x7FFC83E2DA7C" pdb="sqllang.pdb" age="2" guid="C58AAD30-E220-4D32-A11E-C03D11F62263" module="sqllang" rva="0xDA7C" />
<frame id="27" address="0x7FFC83E2EF18" pdb="sqllang.pdb" age="2" guid="C58AAD30-E220-4D32-A11E-C03D11F62263" module="sqllang" rva="0xEF18" />
<frame id="28" address="0x7FFC83E2E5D1" pdb="sqllang.pdb" age="2" guid="C58AAD30-E220-4D32-A11E-C03D11F62263" module="sqllang" rva="0xE5D1" />
<frame id="29" address="0x7FFC84C4D72E" pdb="sqllang.pdb" age="2" guid="C58AAD30-E220-4D32-A11E-C03D11F62263" module="sqllang" rva="0xE2D72E" />
<frame id="30" address="0x7FFC84C751EF" pdb="sqllang.pdb" age="2" guid="C58AAD30-E220-4D32-A11E-C03D11F62263" module="sqllang" rva="0xE551EF" />
<frame id="31" address="0x7FFC84C75A4A" pdb="sqllang.pdb" age="2" guid="C58AAD30-E220-4D32-A11E-C03D11F62263" module="sqllang" rva="0xE55A4A" />
<frame id="32" address="0x7FFC83E38243" pdb="sqllang.pdb" age="2" guid="C58AAD30-E220-4D32-A11E-C03D11F62263" module="sqllang" rva="0x18243" />
<frame id="33" address="0x7FFC83E2B34F" pdb="sqllang.pdb" age="2" guid="C58AAD30-E220-4D32-A11E-C03D11F62263" module="sqllang" rva="0xB34F" />
<frame id="34" address="0x7FFC83E2B470" pdb="sqllang.pdb" age="2" guid="C58AAD30-E220-4D32-A11E-C03D11F62263" module="sqllang" rva="0xB470" />
<frame id="35" address="0x7FFC86CA84DB" pdb="SqlDK.pdb" age="2" guid="EE37F036-82E0-483A-9A70-6A5D6C23610C" module="SqlDK" rva="0x84DB" />
</stackFrames>
<executionStack>
<frame procname="adhoc" line="1" stmtstart="390" stmtend="632" sqlhandle="0x020000004415d703bb1554faf0fea77efb23bd399a83affe0000000000000000000000000000000000000000">
unknown </frame>
<frame procname="unknown" line="1" sqlhandle="0x0000000000000000000000000000000000000000000000000000000000000000000000000000000000000000">
unknown </frame>
</executionStack>
<inputbuf>
(@Version nvarchar(7),@TestAttemptId uniqueidentifier,@TestFormName nvarchar(1),@ActorId uniqueidentifier,@TimeStamp datetime,@Id uniqueidentifier,@UpdatedOn datetime,@UpdatedBy uniqueidentifier)UPDATE [dbo].[TestAttempts] Set [TestFormName]=@TestFormName,[UpdatedOn]=@UpdatedOn,[UpdatedBy]=@UpdatedBy Where [Id]=@Id </inputbuf>
</process>
</process-list>
<resource-list>
<keylock hobtid="72057594060865536" dbid="8" objectname="Beta.dbo.TestAttempts" indexname="PK_TestAttempts" id="lock238c4c3e880" mode="X" associatedObjectId="72057594060865536">
<owner-list>
<owner id="process2fa8801fc28" mode="X" />
</owner-list>
<waiter-list>
<waiter id="process239bfc75088" mode="U" requestType="wait" />
</waiter-list>
</keylock>
<keylock hobtid="72057594060865536" dbid="8" objectname="Beta.dbo.TestAttempts" indexname="PK_TestAttempts" id="lock2bc4724f980" mode="X" associatedObjectId="72057594060865536">
<owner-list>
<owner id="process239bfc75088" mode="X" />
</owner-list>
<waiter-list>
<waiter id="process2fa8801fc28" mode="U" requestType="wait" />
</waiter-list>
</keylock>
</resource-list>
</deadlock>
I've also pasted a query plan here (Note it's not the exact plan from the executions that caused the deadlocks; they are no longer available. It's a plan that results on the same database from running the same update statement):
UPDATE
statements being run inside a transaction of multiple of them, or other statements? If so please include all code that runs in the same transaction.lasttranstarted > lastbatchcompleted > lastbatchstarted
so other statement(s) are running in the transactions prior to the deadlocking ones