37

Using SQL Server 2005.

I am performing a huge DELETE FROM with no where clauses. It's basically equivalent to a TRUNCATE TABLE statement - except I'm not allowed to use TRUNCATE. The problem is the table is huge - 10 million rows, and it takes over an hour to complete. Is there any way of making it faster without:

  • Using Truncate
  • Disabling or dropping indexes?

The t-log is already on a separate disk.

Any suggestions welcome!

  • 2
    If you will be doing this a lot, consider partitioning the table – Gaius Mar 16 '11 at 20:41
  • 1
    Can you not use TRUNCATE because there are FK constraints referencing the table? – Nick Chammas Aug 10 '11 at 15:56
39

What you can do is batch deletes like this:

SELECT 'Starting' --sets @@ROWCOUNT
WHILE @@ROWCOUNT <> 0
    DELETE TOP (xxx) MyTable

Where xxx is, say, 50000

A modification of this, if you want to remove a very high percentage of rows...

SELECT col1, col2, ... INTO #Holdingtable
           FROM MyTable WHERE ..some condition..

SELECT 'Starting' --sets @@ROWCOUNT
WHILE @@ROWCOUNT <> 0
    DELETE TOP (xxx) MyTable WHERE ...

INSERT MyTable (col1, col2, ...)
           SELECT col1, col2, ... FROM #Holdingtable
  • 3
    @tuseau: each delete requires some log space in case of error, to rollback. A 50k row delete takes less resources/space than 10m row delete. Of course, log backups still run etc and take space but it's easier on the server to lots of small batches than mucking big one. – gbn Mar 15 '11 at 14:29
  • 1
    Thanks, the batch delete helps a bit, I guess it's the best option. – tuseau Mar 16 '11 at 9:36
  • 2
    @Phil Helmer: if the batch delete is in a transaction then there is no gain is using it. Otherwise each log write is smaller which is, simply, an easier load – gbn Apr 21 '11 at 7:36
  • 1
    One further comment: the batch delete helps enormously, and takes deleting 20 million rows down from 1 hour 42 min to 3 min - BUT make sure the table has a clustered index! If it's a heap, the TOP clause creates a sort in the execution plan which negates any improvement. Seems obvious afterwards. – tuseau Apr 21 '11 at 9:18
  • 2
    @Noumenon: It ensures @@ROWCOUNT is 1 – gbn Aug 4 '15 at 7:45
21

You could use the TOP clause to get this done easily:

WHILE (1=1)
BEGIN
    DELETE TOP(1000) FROM table
    IF @@ROWCOUNT < 1 BREAK
END
  • The curly brackets format your code – gbn Mar 15 '11 at 15:10
  • @gbn That is on SO. here it is still 101 010. – bernd_k Mar 15 '11 at 19:12
7

I agree with the suggestions to batch your deletes into managable chunks if you aren't able to use TRUNCATE, and I like the drop/create suggestion for it's originality, but I'm curious about the following comment in your question:

It's basically equivalent to a TRUNCATE TABLE statement - except I'm not allowed to use TRUNCATE

I'm guessing the reason for this restriction has to do with the security that needs to be granted to directly truncate a table and the fact that it would allow you to truncate tables other than the one you are concerned with.

Assuming that is the case, I'm wondering if having a stored procedure created that uses TRUNCATE TABLE and uses "EXECUTE AS" would be considered a viable alternative to giving security rights necessary to truncate the table directly.

Hopefully, this would give you the speed you need while also addressing the security concerns that your company may have with adding your account to the db_ddladmin role.

Another advantage of using a stored procedure this way is that the stored procedure itself could be locked down so that only specific account(s) are allowed to use it.

If for some reason this is not an acceptable solution and your need to have the data in this table removed is something that needs to be done once a day/hour/etc, I would request that a SQL Agent job was created to truncate the table at a scheduled time each day.

Hope this helps!

5

Except truncate.. only delete in batches can help you.

You could drop the table and recreate it, with all constraints and indexes, off course. In Management Studio you have the option to script a table to drop and create, so it should be a trivial option. But this only if you're allowed to do DDL actions, which I see it's not really an option.

  • Because the application is designed for concurrent operations, altering the structure (DDL) and using truncate are not options... I guess batch delete is the best available. Thanks though. – tuseau Mar 16 '11 at 9:35
1

Since this question is such an important reference I'm posting this code that really helped me understand deleting with loops and also messaging within a loop to track progress.

The query is modified from this duplicate question. Credit to @RLF for the query base.

CREATE TABLE #DelTest (ID INT IDENTITY, name NVARCHAR(128)); -- Build the test table
INSERT INTO #DelTest (name) SELECT name FROM sys.objects;  -- fill from system DB
SELECT COUNT(*) TableNamesContainingSys FROM #deltest WHERE name LIKE '%sys%'; -- check rowcount
go
DECLARE @HowMany INT;
DECLARE @RowsTouched INT;
DECLARE @TotalRowCount INT;
DECLARE @msg VARCHAR(100);
DECLARE @starttime DATETIME 
DECLARE @currenttime DATETIME 

SET @RowsTouched = 1; -- Needs to be >0 for loop to start
SET @TotalRowCount=0  -- Total rows deleted so far is 0
SET @HowMany = 5;     -- Variable to choose how many rows to delete per loop
SET @starttime=GETDATE()

WHILE @RowsTouched > 0
BEGIN
   DELETE TOP (@HowMany)
   FROM #DelTest 
   WHERE name LIKE '%sys%';

   SET @RowsTouched = @@ROWCOUNT; -- Rows deleted this loop
   SET @TotalRowCount = @TotalRowCount+@RowsTouched; -- Increment Total rows deleted count
   SET @currenttime = GETDATE();
   SELECT @msg='Deleted ' + CONVERT(VARCHAR(9),@TotalRowCount) + ' Records. Runtime so far is '+CONVERT(VARCHAR(30),DATEDIFF(MILLISECOND,@starttime,@currenttime))+' milliseconds.'
   RAISERROR(@msg, 0, 1) WITH NOWAIT;  -- Print message after every loop. Can't use the PRINT function as SQL buffers output in loops.  

END; 
SELECT COUNT(*) TableNamesContainingSys FROM #DelTest WHERE name LIKE '%sys%'; -- Check row count after loop finish
DROP TABLE #DelTest;

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.