3

We have a queue organized with a table on MS SQL Server 2014:

CREATE TABLE [queue].[Messages](
    [Id] [bigint] IDENTITY(1,1) NOT NULL,
    [Status] [tinyint] NOT NULL,
    [Data] [nvarchar](max) NOT NULL,
    [CreationDateUtc] [datetime] NOT NULL
)

CREATE NONCLUSTERED INDEX [IX_Messages_Status] ON [queue].[Messages]
([Status] ASC)

Stored procedure to send message:

ALTER PROCEDURE [queue].[SendMessage]
    @MessageBody NVARCHAR(MAX),
AS
BEGIN
    SET NOCOUNT ON;

    INSERT INTO [queue].[Messages]
    (
        Data,
    )
    VALUES
    (
       @MessageBody,
    )
END

And a stored procedure to receive messages:

ALTER PROCEDURE [queue].[ReceiveMessage]
AS
BEGIN
    SET NOCOUNT ON;

    UPDATE TOP (1) m 
    SET
       m.Status = 2,
       m.StartDateUtc = GETUTCDATE()
    OUTPUT
       INSERTED.*
    FROM [queue].[Messages] m WITH (UPDLOCK, READPAST)
    WHERE m.Status = 1
END

This stored procedure executes every second to check new messages in queue. Very often we get blocks (PAGELATCH_EX) on this query even if the queue is empty: enter image description here

First idea was - diff-backups, but they are scheduled at different times: enter image description here

Other queries don't make blocks so often.

  1. Am I right that this is due to UPDLOCK and READPAST hints?
  2. Are there other reasons?
10
  • 2
    So you have a heap and you have UPDATE TOP (1) with no discernible ORDER BY - you know that your UPDATE is arbitrary and non-deterministic, right? So it might be trying to update the latest row instead of the earliest row? You might want to change your implementation a little bit. Commented Oct 13, 2015 at 15:02
  • @AaronBertrand yes, I know about this. This assumption was made consciously, because I don't need strong order for handling messages now. And I'll change this behavior for future tasks soon. Thank you for the link, it will help
    – Backs
    Commented Oct 13, 2015 at 15:22
  • When rows are deleted from a heap, that space is not automatically freed for re-use. This can result in a lot of unusable space in your database, depending on how many rows are inserted and deleted into this table.
    – Hannah Vernon
    Commented Oct 13, 2015 at 16:17
  • 1
    You didn't mention anything about deleting the rows, so if there's a lot of rows, maybe the status index should be filtered? Haven't tried that but it would sound logical to only index rows with status = 1
    – James Z
    Commented Oct 13, 2015 at 16:53
  • 1
    Because I was quite sure it wasn't actually a good answer. Please take Remus' advice!
    – Hannah Vernon
    Commented Oct 14, 2015 at 13:01

1 Answer 1

4

This is because you use a Status field for messages and leave unreceiveable messages in the queue. Retaining messages (keeping them after process, with a different status) leads to queue bloat, contention, poor plan choices and general slow processing.

Follow Using tables as Queues and do destructive message processing:

DELETE TOP (1) 
FROM [queue].[Messages] m WITH (ROWLOCK, READPAST)
OUTPUT deleted.*;

Also drop all indexes on the queue. The linked article has details how to achieve fancier behavior like FIFO or Pending.

Use queues only for events, not for state.

3
  • If queue messages must be retained, they can be moved into a table on different filegroup, preferably on a different set of disks, correct?
    – Hannah Vernon
    Commented Oct 14, 2015 at 13:04
  • 1
    @MaxVernon that's what I mean by 'keep events, not state'. After the event is consumed (ie. message is deleted), if you wish to keep the history (=state), keep it whenever you wish. Commented Oct 14, 2015 at 13:31
  • I'll follow your advice in future. But now I've just added filtered index, because I can't change message processing code right now.
    – Backs
    Commented Oct 15, 2015 at 3:47

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.