Return to Revisions

The behavior you are seeing here is due, in a general sense, to the fact that the Unicode Collation Algorithm (UCA) allows for complex, multi-level sorting. More specifically:

1. Sorting is not Comparison:

Determining whether two strings are the same or different is fairly straight forward (given a particular locale/language and set of sensitivities). But determining the order of 2 or more strings can be highly complex.

1. Sorting is done in a series of steps, with each step applied to the entire string, not character by character:

2. Standard: sort base characters (regardless of accent and case differences)

3. IF Accent-sensitive, apply accent / diacritic weights

4. IF Case-sensitive, apply casing weights

When you sort by col1 (single character), it first determines that all characters have the same weight since they are all "e". Next, it applies the accent / diacritic weights. There are no casing differences so the third step wouldn't change anything. So the only differences are in step 2, which is why there is a preferred order for those rows based on col1.

When you sort by col2 (two characters), it first determines that each row has a different weight since both characters are used to determine the sort weight (e.g. "ea", "eb", etc). Next, it applies the accent / diacritic weights. There are no casing differences so the third step wouldn't change anything. So there are differences in steps 1 and 2 this time. But since the differences in step 1 have already been applied to each string before the weights of step 2 are considered, the weights from step 2 don't have any effect on the ordering; they would only apply if weights from step 1 for two or more rows were the same.

The following adaptation of the sample code from the question hopefully illustrates the sorting behavior described above. I added some additional rows and an additional column to help show the impact of the Collation being case-sensitive (since the original sample data is all lower-case):

SETUP

USE [tempdb];

-- DROP TABLE dbo.OddSort;
CREATE TABLE dbo.OddSort
(
    id INT IDENTITY(1,1) PRIMARY KEY,
    col1 NVARCHAR(5) COLLATE Latin1_General_100_CS_AS,
    col2 NVARCHAR(5) COLLATE Latin1_General_100_CS_AS,
    col3 NVARCHAR(5) COLLATE Latin1_General_100_CS_AS
);
GO

INSERT dbo.OddSort (col1, col2, col3)
VALUES (N'e', N'eA', N'è 1')
     , (N'ê', N'êE', N'ê 5')
     , (N'é', N'éH', N'a 9')
     , (N'ë', N'ëC', N'e 2')
     , (N'E', N'EG', N'E 7')
     , (N'Ë', N'ëh', N'Ë 0')
     , (N'è', N'èD', N'é 4')
     , (N'é', N'éB', N'ē 3')
     , (N'ë', N'ëH', N'e 8')
     , (N'ē', N'ēF', N'ë 6');

TEST 1

SELECT [id], [col1], UNICODE([col1]) AS [CodePoint]
FROM dbo.OddSort 
ORDER BY col1;

Returns:

id    col1    CodePoint
1     e       101
5     E       69
8     é       233
3     é       233
7     è       232
2     ê       234
4     ë       235
9     ë       235
6     Ë       203
10    ē       275

What we can see in the results above:

1. The Code Point is not determining the sort order
2. The non-accented characters are sorted before the accented characters (within the same letter: f would still come after all of these). Clearly, accent weights are applied before case weights.
3. Lower-case sorts before upper-case within the same accented (or non-accented) character (i.e. the e then E, and the ë then Ë). This tailoring is used by most of the Windows Collations, while most of the SQL Server Collations sort upper-case first.

TEST 2

SELECT [id], [col2]
FROM dbo.OddSort 
ORDER BY col2;

Returns:

id    col2
1     eA
8     éB
4     ëC
7     èD
2     êE
10    ēF
5     EG
3     éH
6     ëh
9     ëH

TEST 3

SELECT [id], [col3]
FROM dbo.OddSort 
ORDER BY col3;

Returns:

id    col3
3     a 9
6     Ë 0
1     è 1
4     e 2
8     ē 3
7     é 4
2     ê 5
10    ë 6
5     E 7
9     e 8

Additional notes:

1. With regarding to getting the exact rules, that is not as easy as it should be. The problem with getting concrete explanations of these rules is that the Unicode sorting rules, while definitely documented, are a recommendation. It is up to vendors, such as Microsoft, to implement those recommendations. Microsoft did not implement the recommendations exactly as stated in the Unicode documentation so there is a disconnect there (similar to how neither the HTML or CSS specifications are implemented as completely, nor even in the same way, across vendors). Then, there are different versions of the Windows Collations (you are using version 100 which came out with SQL Server 2008) and that is tied to a Unicode version that is much older than the current version of Unicode or of the ICU Collation demo is using. Still, the UCA documentation linked above is a good place to start.

2. Since the datatype is NVARCHAR, we are only concerned with Unicode Code Points and sorting algorithms, hence the use of the UNICODE() function in TEST 1. While Code Pages are specified by most Collations, they only pertain to VARCHAR data. Meaning, while Code Page 1252 is specified by the Latin1_General* series of Collations, that can be ignored here.