7

Assuming a database using full recovery model, when a record is written in SQL Server (by INSERT / UPDATE etc) write ahead logging will ensure the change is written to the log file before modifying the data page.

Both the log and data page entries are made in RAM and committed to disk later by a Checkpoint.

If there is a system crash (power loss for the sake of argument) what will happen to dirty pages (IE data that is changed in RAM but not committed to disk) as the contents of RAM do not survive system restarts, is this data lost?

EDIT

After some testing, I can see the dirty pages are not lost, but I am not sure why:

using this tutorial

create a test database

CREATE DATABASE DirtyPagesDB
GO
USE DirtyPagesDB
GO

turn off automatic checkpoints

DBCC TRACEON(3505, -1);
DBCC TRACESTATUS();

create a table, insert some data and issue a checkpoint:

CREATE TABLE t1 (Speaker_Bio CHAR(8000))
GO
INSERT INTO t1 VALUES ('SQL'),('Authority')
GO
CHECKPOINT

confirm no dirty pages

-- Get the rows of dirtied pages
SELECT
database_name = d.name,
OBJECT_NAME =
CASE au.TYPE
WHEN 1 THEN o1.name
WHEN 2 THEN o2.name
WHEN 3 THEN o1.name
END,
OBJECT_ID =
CASE au.TYPE
WHEN 1 THEN p1.OBJECT_ID
WHEN 2 THEN p2.OBJECT_ID
WHEN 3 THEN p1.OBJECT_ID
END,
index_id =
CASE au.TYPE
WHEN 1 THEN p1.index_id
WHEN 2 THEN p2.index_id
WHEN 3 THEN p1.index_id
END,
bd.FILE_ID,
bd.page_id,
bd.page_type,
bd.page_level
FROM sys.dm_os_buffer_descriptors bd
INNER JOIN sys.databases d
ON bd.database_id = d.database_id
INNER JOIN sys.allocation_units au
ON bd.allocation_unit_id = au.allocation_unit_id
LEFT JOIN sys.partitions p1
ON au.container_id = p1.hobt_id
LEFT JOIN sys.partitions p2
ON au.container_id = p2.partition_id
LEFT JOIN sys.objects o1
ON p1.OBJECT_ID = o1.OBJECT_ID
LEFT JOIN sys.objects o2
ON p2.OBJECT_ID = o2.OBJECT_ID
WHERE is_modified = 1
AND d.name = 'DirtyPagesDB'
AND
(
o1.name = 't1'
OR o2.name = 't1'
);
GO

confirm time of last checkpoint

SELECT  f1.[Checkpoint Begin], f2.[Checkpoint End]
FROM    fn_dblog(NULL, NULL) f1
        JOIN fn_dblog(NULL, NULL) f2
             On f1.[Current LSN] = f2.[Previous LSN]
WHERE   f2.Operation IN (N'LOP_BEGIN_CKPT', N'LOP_END_CKPT');

Add more rows

INSERT INTO t1 VALUES ('SQL'),('Authority')

Use query above to confirm that there were dirty pages

Kill the SQL Server task from task manager to simulate a power off.

Start the service

Re-run the command above to get the last checkpoint time, it was the same (IE no checkpoints have run other than the one we did manually)

SELECTed from the table t1 and all four records were there

  • 1
    Log records are always written first to guarantee durability. Dirty pages are written via the Lazy Writer and Checkpoint processes to reduce recovery time. See docs.microsoft.com/en-us/sql/2014-toc/…. – Dan Guzman Aug 19 '18 at 22:02
  • If the new log records are still in RAM and not written to disk at the time of failure, those changes cannot be recovered? – SEarle1986 Aug 19 '18 at 22:10
  • 5
    Log records containing data modified by the transaction are physically written at the time of commit to guarantee durability. If only in memory, that means the transaction has not committed and will be rolled back during database recovery. That way, the database will be consistent after recovery. – Dan Guzman Aug 19 '18 at 22:40
15

Both the log and data page entries are made in RAM and committed to disk later by a Checkpoint.

This statement is not completely true. It is correct that data pages are written to disk by Checkpoint (and Lazy Writer). Log records, however, are physically written to disk when the transaction is committed to guarantee transaction durability. A committed transaction data will never be only memory resident (barring delayed durability).

All data modifications are first written to the log (write-ahead logging) and dirty pages written afterward. Pages and log records may include both committed and uncommitted data on disk.

Regardless of the recovery model, SQL Server scans the log during crash recovery to the last checkpoint, rolls forward all data modifications from that point forward, and finally rolls back uncommitted transactions.

  • Thanks, I wasn't aware the log file is always written to disk on commit. I've read a fair few articles on this including some Microsoft ones and none of them made that as clear as you have – SEarle1986 Aug 20 '18 at 11:06
  • @SEarle1986, glad this answer helped your understanding. Buried in the documentation I referenced are the key points I summarized: "SQL Server has logic that prevents a dirty page from being flushed before the associated log record is written. Log records are written to disk when the transactions are committed." – Dan Guzman Aug 20 '18 at 11:16
  • Dan (and @SEarle1986 ): Is this the REDO operation? And, are there any cases where this step doesn't happen, is skipped, etc? – Solomon Rutzky Aug 20 '18 at 15:24
  • @SolomonRutzky, yes roll forward and REDO are the same thing in this context. The only case where this isn't needed during crash recovery is when there are no changes since the last checkpoint. – Dan Guzman Aug 20 '18 at 22:27
  • What does "rolls forward all data modifications from that point forward" mean for minimally logged transactions? – Joe Obbish Aug 20 '18 at 23:05

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.