29

I'm a fan of surrogate keys. There is a risk my findings are confirmation biased.

Many questions I've seen both here and at http://stackoverflow.com use natural keys instead of surrogate keys based on IDENTITY() values.

My background in computer systems tells me performing any comparative operation on an integer will be faster than comparing strings.

This comment made me question my beliefs, so I thought I would create a system to investigate my thesis that integers are faster than strings for use as keys in SQL Server.

Since there is likely to be very little discernible difference in small datasets, I immediately thought of a two table setup where the primary table has 1,000,000 rows and the secondary table has 10 rows for each row in the primary table for a total of 10,000,000 rows in the secondary table. The premise of my test is to create two sets of tables like this, one using natural keys and one using integer keys, and run timing tests on a simple query like:

SELECT *
FROM Table1
    INNER JOIN Table2 ON Table1.Key = Table2.Key;

The following is the code I created as a test bed:

USE Master;
IF (SELECT COUNT(database_id) FROM sys.databases d WHERE d.name = 'NaturalKeyTest') = 1
BEGIN
    ALTER DATABASE NaturalKeyTest SET SINGLE_USER WITH ROLLBACK IMMEDIATE;
    DROP DATABASE NaturalKeyTest;
END
GO
CREATE DATABASE NaturalKeyTest 
    ON (NAME = 'NaturalKeyTest', FILENAME = 
        'C:\SQLServer\Data\NaturalKeyTest.mdf', SIZE=8GB, FILEGROWTH=1GB) 
    LOG ON (NAME='NaturalKeyTestLog', FILENAME = 
        'C:\SQLServer\Logs\NaturalKeyTest.mdf', SIZE=256MB, FILEGROWTH=128MB);
GO
ALTER DATABASE NaturalKeyTest SET RECOVERY SIMPLE;
GO
USE NaturalKeyTest;
GO
CREATE VIEW GetRand
AS 
    SELECT RAND() AS RandomNumber;
GO
CREATE FUNCTION RandomString
(
    @StringLength INT
)
RETURNS NVARCHAR(max)
AS
BEGIN
    DECLARE @cnt INT = 0
    DECLARE @str NVARCHAR(MAX) = '';
    DECLARE @RandomNum FLOAT = 0;
    WHILE @cnt < @StringLength
    BEGIN
        SELECT @RandomNum = RandomNumber
        FROM GetRand;
        SET @str = @str + CAST(CHAR((@RandomNum * 64.) + 32) AS NVARCHAR(MAX)); 
        SET @cnt = @cnt + 1;
    END
    RETURN @str;
END;
GO
CREATE TABLE NaturalTable1
(
    NaturalTable1Key NVARCHAR(255) NOT NULL 
        CONSTRAINT PK_NaturalTable1 PRIMARY KEY CLUSTERED 
    , Table1TestData NVARCHAR(255) NOT NULL 
);
CREATE TABLE NaturalTable2
(
    NaturalTable2Key NVARCHAR(255) NOT NULL 
        CONSTRAINT PK_NaturalTable2 PRIMARY KEY CLUSTERED 
    , NaturalTable1Key NVARCHAR(255) NOT NULL 
        CONSTRAINT FK_NaturalTable2_NaturalTable1Key 
        FOREIGN KEY REFERENCES dbo.NaturalTable1 (NaturalTable1Key) 
        ON DELETE CASCADE ON UPDATE CASCADE
    , Table2TestData NVARCHAR(255) NOT NULL  
);
GO

/* insert 1,000,000 rows into NaturalTable1 */
INSERT INTO NaturalTable1 (NaturalTable1Key, Table1TestData) 
    VALUES (dbo.RandomString(25), dbo.RandomString(100));
GO 1000000 

/* insert 10,000,000 rows into NaturalTable2 */
INSERT INTO NaturalTable2 (NaturalTable2Key, NaturalTable1Key, Table2TestData)
SELECT dbo.RandomString(25), T1.NaturalTable1Key, dbo.RandomString(100)
FROM NaturalTable1 T1
GO 10 

CREATE TABLE IDTable1
(
    IDTable1Key INT NOT NULL CONSTRAINT PK_IDTable1 
    PRIMARY KEY CLUSTERED IDENTITY(1,1)
    , Table1TestData NVARCHAR(255) NOT NULL 
    CONSTRAINT DF_IDTable1_TestData DEFAULT dbo.RandomString(100)
);
CREATE TABLE IDTable2
(
    IDTable2Key INT NOT NULL CONSTRAINT PK_IDTable2 
        PRIMARY KEY CLUSTERED IDENTITY(1,1)
    , IDTable1Key INT NOT NULL 
        CONSTRAINT FK_IDTable2_IDTable1Key FOREIGN KEY 
        REFERENCES dbo.IDTable1 (IDTable1Key) 
        ON DELETE CASCADE ON UPDATE CASCADE
    , Table2TestData NVARCHAR(255) NOT NULL 
        CONSTRAINT DF_IDTable2_TestData DEFAULT dbo.RandomString(100)
);
GO
INSERT INTO IDTable1 DEFAULT VALUES;
GO 1000000
INSERT INTO IDTable2 (IDTable1Key)
SELECT T1.IDTable1Key
FROM IDTable1 T1
GO 10

The code above creates a database and 4 tables, and fills the tables with data, ready to test. The test code I ran is:

USE NaturalKeyTest;
GO
DECLARE @loops INT = 0;
DECLARE @MaxLoops INT = 10;
DECLARE @Results TABLE (
    FinishedAt DATETIME DEFAULT (GETDATE())
    , KeyType NVARCHAR(255)
    , ElapsedTime FLOAT
);
WHILE @loops < @MaxLoops
BEGIN
    DBCC FREEPROCCACHE;
    DBCC FREESESSIONCACHE;
    DBCC FREESYSTEMCACHE ('ALL');
    DBCC DROPCLEANBUFFERS;
    WAITFOR DELAY '00:00:05';
    DECLARE @start DATETIME = GETDATE();
    DECLARE @end DATETIME;
    DECLARE @count INT;
    SELECT @count = COUNT(*) 
    FROM dbo.NaturalTable1 T1
        INNER JOIN dbo.NaturalTable2 T2 ON T1.NaturalTable1Key = T2.NaturalTable1Key;
    SET @end = GETDATE();
    INSERT INTO @Results (KeyType, ElapsedTime)
    SELECT 'Natural PK' AS KeyType, CAST((@end - @start) AS FLOAT) AS ElapsedTime;

    DBCC FREEPROCCACHE;
    DBCC FREESESSIONCACHE;
    DBCC FREESYSTEMCACHE ('ALL');
    DBCC DROPCLEANBUFFERS;
    WAITFOR DELAY '00:00:05';
    SET @start = GETDATE();
    SELECT @count = COUNT(*) 
    FROM dbo.IDTable1 T1
        INNER JOIN dbo.IDTable2 T2 ON T1.IDTable1Key = T2.IDTable1Key;
    SET @end = GETDATE();
    INSERT INTO @Results (KeyType, ElapsedTime)
    SELECT 'IDENTITY() PK' AS KeyType, CAST((@end - @start) AS FLOAT) AS ElapsedTime;

    SET @loops = @loops + 1;
END
SELECT KeyType, FORMAT(CAST(AVG(ElapsedTime) AS DATETIME), 'HH:mm:ss.fff') AS AvgTime 
FROM @Results
GROUP BY KeyType;

These are the results:

enter image description here

Am I doing something wrong here, or are INT keys 3 times faster than 25 character natural keys?

Note, I've written a follow-up question here.

2
  • 9
    I think the performance gain @MikeSherrill'Catcall' was getting at is that you don't actually need the join against the "lookup" table when you use a natural key. Compare a query to get the lookup value with a join, with a query where the value is already stored in the main table. You might get a different "winner" depending on the natural key length and number of rows in the lookup table. Commented Sep 29, 2013 at 5:43
  • 4
    What @MikaelEriksson said plus the cases when you have a join between more than 2 tables (say 4) where with the surrogates you'll have to join tables A to D through B and C while with natural keys you could join A to D directly Commented Sep 29, 2013 at 7:15

3 Answers 3

23

In general, SQL Server uses B+Trees for indexes. The expense of an index seek is directly related to the length of the key in this storage format. Hence, a surrogate key usually outperforms a natural key on index seeks.

SQL Server clusters a table on the primary key by default. The clustered index key is used to identify rows, so it gets added as included column to every other index. The wider that key, the larger every secondary index.

Even worse, if the secondary indexes are not explicitly defined as UNIQUE the clustered index key automatically becomes part of the key of each of those. That usually applies to most indexes, as usually indexes are declared as unique only when the requirement is to enforce uniqueness.

So if the question is, natural versus surrogate clustered index, the surrogate will almost always win.

On the other hand, you are adding that surrogate column to the table making the table in itself bigger. That will cause clustered index scans to get more expensive. So, if you have only very few secondary indexes and your workload requires to look at all (or most of the) rows often, you actually might be better of with a natural key saving those few extra bytes.

Finally, natural keys often make it easier to understand the data model. While using more storage space, natural primary keys lead to natural foreign keys which in turn increase local information density.

So, as so often in the database world, the real answer is "it depends". And - always test in your own environment with realistic data.

0
11

I believe, that the best lies in the middle.

Natural keys overview:

  1. They are make data model more obvious because they are came from subject area, and not from somebody's head.
  2. Simple keys (one column, between CHAR(4) and CHAR(20)) are saving some extra bytes, but you need to watch for their consistency (ON UPDATE CASCADE becomes critical for those keys, that might be changed).
  3. A lot of cases, when natural keys are complex: consists of two or more columns. If such key might migrate to another entity as a foreing key, then it will add data overhead (indices and data columns might become large) and performance loose.
  4. If key is a large string, then it probably always will loose to an integer key, because simple search condition becames a byte array comparison in a database engine, which in most cases is slower, than integer comparison.
  5. If key is a multilanguage string then need to watch the collations also.

Benefits: 1 and 2.

Watchouts: 3, 4 and 5.


Artificial identity keys overview:

  1. You do not need to bother about their creation and handling (in most cases) as this feature handled by database engine. They are unique by default and doesn't take a lot of space. Custom operations like ON UPDATE CASCADE might be ommited, because key values not changing.

  2. They (often) are best candidates for migration as a foreign keys because:

    2.1. consists of one column;

    2.2. using a simple type which has a small weight and acts fast for comparison operations.

  3. For an association entities, which keys are not migrate anywhere, it might become a pure data overhead, as it usefulness is lost. Complex natural primary key (if there are no string columns there) will be more useful.

Benefits: 1 and 2.

Watchouts: 3.


CONCLUSION:

Arificial keys are more maintainable, reliable and fast because they have been designed for this features. But in some cases are not needed. For example, single CHAR(4) column candidate in most cases behaves like INT IDENTITY. So there is another question here also: maintainability + stability or obviousness?

Question "Should I inject an artificial key or not?" always depends on natural key structure:

  • If it contains a large string, then it is slower and will add data overhead if migrating as foreign to another entity.
  • If it consists of multiple columns, then it is slower and will add data overhead if migrating as foreign to another entity.
1
  • 5
    "Custom operations like ON UPDATE CASCADE might be ommited, because key values not changing." The effect of surrogate keys is to make every foreign key reference the equivalent of "ON UPDATE CASCADE". The key doesn't change, but the value it represents does. Commented Sep 30, 2013 at 12:04
8

A key is a logical feature of a database whereas performance is always determined by physical implementation in storage and by physical operations run against that implementation. It's therefore a mistake to attribute performance characteristics to keys.

In this particular example however, two possible implementations of tables and queries are compared to each other. The example does not answer the question being posed in the title here. The comparison being made is of joins using two different datatypes (integer and character) using just one type of index (B-tree). An "obvious" point is that if a hash index or other type of index been used there would quite possibly be no measurable performance difference between the two implementations. There are more fundamental problems with the example however.

Two queries are being compared for performance but the two queries are not logically equivalent because they return different results! A more realistic test would compare two queries returning the same results but using different implementations.

The essential point about a surrogate key is that it is an extra attribute in a table where the table also has "meaningful" key attributes used in the business domain. It is the non-surrogate attributes that are of interest for query results to be useful. A realistic test therefore would compare tables using only natural keys with an alternative implementation having both natural and surrogate keys in the same table. Surrogate keys typically require additional storage and indexing and by definition require additional uniqueness constraints. Surrogates require additional processing to map the external natural key values onto their surrogates and vice versa.

Now compare this potential query:

A.

SELECT t2.NaturalTable2Key, t2.NaturalTable1Key
FROM Table2 t2;

To its logical equivalent if the NaturalTable1Key attribute in Table2 is replaced with the surrogate IDTable1Key:

B.

SELECT t2.NaturalTable2Key, t1.NaturalTable1Key
FROM Table2 t2
INNER JOIN Table1 t1
ON t1.IDTable1Key = t2.IDTable1Key;

Query B requires a join; Query A does not. This is a familiar situation in databases that (over)use surrogates. Queries become needlessly complex and much harder to optimise. Business logic (especially data integrity constraints) becomes more difficult to implement, test and verify.

1
  • This should have been the answer!
    – lmk
    Commented Aug 21, 2023 at 22:34

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.