7

There are two types of ways to perform a PIVOT. Before SQL Server 2005, when PIVOT was introduced, most people did this:

SELECT RateID
              SUM(CASE WHEN RateItemTypeID = 1 THEN UnitPrice ELSE 0 END),
              SUM(CASE WHEN RateItemTypeID = 2 THEN UnitPrice ELSE 0 END),
              SUM(CASE WHEN RateItemTypeID = 3 THEN UnitPrice ELSE 0 END)
              FROM rate_item WHERE _WhereClause_
              GROUP BY RateID

Later, when 2005 introduced PIVOT it became this:

   SELECT RateID, [1], [2], [3]
          FROM PertinentRates -- PertinentRates is a CTE with WHERE clause applied
          PIVOT (SUM(UnitPrice) FOR RateItemTypeID IN ([1], [2], [3])) PVT)

Across SQL Server 2005, 2008 R2, 2012 and 2014 (the versions of SQL Server I've worked with that implement PIVOT), in my experience, it has always been faster than SUM(CASE) or in a few cases equally fast. Are there examples where PIVOT is slower?

I can't give the DDL because it is an example from my work. But the table is pretty simple. In the PIVOT example it is drawing from a CTE whereas the SUM(CASE) is drawing directly from the table. But the SUM(CASE) performs the same drawing from the CTE.

In my work example, the PIVOT comes back in 10 seconds while the SUM(CASE) comes back in 14. Clearly it must be doing something different under the covers. The plans are the same, 50% of total each. PIVOT converted to SUM(CASE) in the query analyzer. Yet SUM(CASE) never comes back in less than 13 seconds, and PIVOT never comes back in over 11 seconds.

I've tried running them back and forth, it doesn't matter the order they are run in. If I run them both from a cold cache they both take longer, but PIVOT is still faster, 12 vs 17 seconds. Can't reproduce on a second server, but that one is considerably better; it's 5 seconds each there with minor variations. PIVOT is a little better, but percentage-wise it doesn't have the same edge as on the first server.

The IO stats, like the query plan, is identical between the two. That is odd, I had kind of expected to see different IO stats, even though I've never looked at them for this particular example.

20

Are there examples where PIVOT is slower?

This is unlikely in simple cases. As Itzik Ben-Gan notes in his SQL Server Pro article, Pivoting Data when looking at the plan for a PIVOT query (emphasis added):

Figure 3 shows the plan for the PIVOT query. As you can see, this plan is very similar to that of the standard solution—so much so that if you look at the properties of the Aggregate operator, under Defined Values, you’ll find that SQL Server constructed CASE expressions behind the scenes:


[Expr1022] = Scalar Operator(SUM(CASE WHEN [InsideTSQL2008].[Sales].[Orders].[shipcity]=N'Barcelona' THEN [InsideTSQL2008].[Sales].[Orders].[freight] ELSE NULL END))

With this in mind, you shouldn’t expect the solution that’s based on the PIVOT operator to perform better than the standard solution. The main benefit in the PIVOT operator at the moment is that it’s less verbose.

For more advanced pivoting requirements that the (non-standard) PIVOT syntax does not directly support, workarounds are needed. These may or may not lead to worse performance compared with CASE, depending on various factors including the skill level of the implementor.

Examples of these problematic cases are covered in Itzik's article, and also well explained in Robert Sheldon's Simple Talk article, Questions About Pivoting Data in SQL Server You Were Too Shy to Ask.

My experience has been that PIVOT and Agg(CASE... generate extremely similar plans with extremely close performance characteristics when both are written optimally. My usual advice is to write queries using whatever syntax feels most natural to you, and to only try rewrites if performance is not acceptable.

Internals

The SQL Server query processor does have a built-in Pivot logical operator (LogOp_Pivot), so it is maybe not quite correct to say that SQL Server rewrites pivots to aggregates and case expressions, at least if we are talking about parsing and compilation activities that take place prior to cost-based optimization (trivial plans are not available for pivot queries).

On the other hand, it is true that the only way the optimizer can implement a query tree containing LogOp_Pivot is via the exploration rule ExpandPivot. This rule expands LogOp_Pivot into a normal grouped aggregate (LogOp_GbAgg) with associated scalar expressions. When this rule is disabled, pivot queries fail to compile.

In practice then, we can say that pivots are always (eventually) 'rewritten' as aggregates and scalar expressions before an executable plan can be produced.

Anyway, the result of the rewrite to LogOp_GbAgg is converted to the physical operators needed for an executable plan by the regular group-by aggregate implementation rules GbAggToHS (hash) or GbAggToStrm (stream).

As a side note, the reason 'manual pivots' (aggregates on case expressions) have an extra Compute Scalar below the aggregate is that the case expressions are pushed toward the leaf level of the query tree during Project Normalization (an early stage of compilation, before cost-based optimization).

Queries that use the PIVOT syntax do not have this because the expressions are not created until ExpandPivot runs during cost-based optimization. At the (earlier) time Project Normalization runs, the query tree still has LogOp_Pivot elements, so there are no projections to push down, and the case expressions typically end up inside the hash or stream aggregate.

There is typically no advantage in avoiding the Compute Scalar, since from SQL Server 2005 onward, expression evaluation is normally deferred until the result is required by a later operator. In this case, evaluation of the case expressions is deferred until the aggregate (hash or stream) requires it.

12

Repeating the tests from Cross Tabs and Pivots, Part 1 – Converting Rows to Columns - By Jeff Moden, 2010/08/06 (first published: 2008/08/19) on rextester

Unfortunately I can not access statistics for IO, Time or execution plans on rextester, but it has the unique benefit of being a common test environment that anyone here can tinker with and examine. I realize this still leaves something to be desired in regards to digging in and investigating exactly what is going on, but it I would say being able to share a testing environment is an important facet for this discussion.


Simple

rextester: http://rextester.com/BAZMGJ69528

This one was added for @MartinSmith and while the queries are pulled from the same article, it was not in the original tests like this:

create table #timer (what varchar(64), ended datetime);
insert into #timer values ('Start',getdate());
go

SELECT TOP 400000 --<<Look!  Change this number for testing different size tables
        RowNum       = IDENTITY(INT,1,1),
        Company      = CHAR(ABS(CHECKSUM(NEWID()))%2+65)
                     + CHAR(ABS(CHECKSUM(NEWID()))%2+65)
                     + CHAR(ABS(CHECKSUM(NEWID()))%2+65),
        Amount       = CAST(ABS(CHECKSUM(NEWID()))%1000000/100.0 AS MONEY),
        Quantity     = ABS(CHECKSUM(NEWID()))%50000+1,
        Date         = CAST(RAND(CHECKSUM(NEWID()))*3653.0+36524.0 AS DATETIME),
        Year         = CAST(NULL AS SMALLINT),
        Quarter      = CAST(NULL AS TINYINT)
   INTO #SomeTable3
   FROM Master.sys.SysColumns t1
  CROSS JOIN
        Master.sys.SysColumns t2 

--===== Fill in the Year and Quarter columns from the Date column
 UPDATE #SomeTable3
    SET Year    = DATEPART(yy,Date),
        Quarter = DATEPART(qq,Date)

--===== A table is not properly formed unless a Primary Key has been assigned
     -- Takes about 1 second to execute.
  ALTER TABLE #SomeTable3
        ADD PRIMARY KEY CLUSTERED (RowNum)
CREATE NONCLUSTERED INDEX IX_#SomeTable3_CoverYear 
    ON dbo.#SomeTable3 (Year)
       INCLUDE (Amount, Quantity, Quarter) 

create statistics syear on #sometable3(year) with fullscan, norecompute;
create statistics syearquarter on #sometable3(year,quarter) with fullscan, norecompute;
GO
insert into #timer values ('Finished Loading Test Data',getdate());
go
--===== Simple Pivot 
 SELECT Year, 
        COALESCE([1],0) AS [1st Qtr],
        COALESCE([2],0) AS [2nd Qtr],
        COALESCE([3],0) AS [3rd Qtr],
        COALESCE([4],0) AS [4th Qtr],
        COALESCE([1],0) + COALESCE([2] ,0) + COALESCE([3],0) + COALESCE([4],0) AS Total
   into #SimplePivot_prep
   FROM (SELECT Year, Quarter,Amount FROM #SomeTable3)  AS src 
  PIVOT (SUM(Amount) FOR Quarter IN ([1],[2],[3],[4])) AS pvt 
go
--===== Simple Cross Tab
 SELECT Year,
        SUM(CASE WHEN Quarter = 1 THEN Amount ELSE 0 END) AS [1st Qtr],
        SUM(CASE WHEN Quarter = 2 THEN Amount ELSE 0 END) AS [2nd Qtr],
        SUM(CASE WHEN Quarter = 3 THEN Amount ELSE 0 END) AS [3rd Qtr],
        SUM(CASE WHEN Quarter = 4 THEN Amount ELSE 0 END) AS [4th Qtr],
        SUM(Amount) AS Total
   into #simpleCrossTab_prep
   FROM #SomeTable3
  GROUP BY Year
go
--insert into #timer values ('Simple Cross Tab',getdate());
go
--=====--
insert into #timer values ('Finished Prep',getdate());
go
--=====--
--===== Simple Pivot
 SELECT Year, 
        COALESCE([1],0) AS [1st Qtr],
        COALESCE([2],0) AS [2nd Qtr],
        COALESCE([3],0) AS [3rd Qtr],
        COALESCE([4],0) AS [4th Qtr],
        COALESCE([1],0) + COALESCE([2] ,0) + COALESCE([3],0) + COALESCE([4],0) AS Total
   into #SimplePivot
   FROM (SELECT Year, Quarter,Amount FROM #SomeTable3)  AS src 
  PIVOT (SUM(Amount) FOR Quarter IN ([1],[2],[3],[4])) AS pvt 
go
insert into #timer values ('Simple Pivot',getdate());
go
--=====--
--===== Simple Cross Tab
 SELECT Year,
        SUM(CASE WHEN Quarter = 1 THEN Amount ELSE 0 END) AS [1st Qtr],
        SUM(CASE WHEN Quarter = 2 THEN Amount ELSE 0 END) AS [2nd Qtr],
        SUM(CASE WHEN Quarter = 3 THEN Amount ELSE 0 END) AS [3rd Qtr],
        SUM(CASE WHEN Quarter = 4 THEN Amount ELSE 0 END) AS [4th Qtr],
        SUM(Amount) AS Total
   into #simpleCrossTab
   FROM #SomeTable3
  GROUP BY Year
go 
insert into #timer values ('Simple Cross Tab',getdate());
go
--=====--
select 
    o.what
  , started=isnull(convert(varchar(30),x.ended),o.ended)
  , ended=convert(varchar(30),o.ended)
  , DurationInMs=datediff(millisecond,x.ended,o.ended)
from #timer o
  outer apply (select top 1 ended from #timer i where i.ended < o.ended order by i.ended desc) as x

returns:

+----------------------------+---------------------+---------------------+--------------+
|            what            |       started       |        ended        | DurationInMs |
+----------------------------+---------------------+---------------------+--------------+
| Start                      | Feb 19 2017  7:13PM | Feb 19 2017  7:13PM | NULL         |
| Finished Loading Test Data | Feb 19 2017  7:13PM | Feb 19 2017  7:13PM | 7210         |
| Finished Prep              | Feb 19 2017  7:13PM | Feb 19 2017  7:13PM | 700          |
| Simple Pivot               | Feb 19 2017  7:13PM | Feb 19 2017  7:13PM | 340          |
| Simple Cross Tab           | Feb 19 2017  7:13PM | Feb 19 2017  7:13PM | 386          |
+----------------------------+---------------------+---------------------+--------------+

The remainder all test limitations in the pivot syntax where a single cross tab query can accomplish that which would require multiple pivots.

Normal

rextester: http://rextester.com/UVZE87903

create table #timer (what varchar(64), ended datetime);
insert into #timer values ('Start',getdate());
go

 SELECT TOP 300000 --<<Look!  Change this number for testing different size tables
        RowNum       = IDENTITY(INT,1,1),
        Company      = CHAR(ABS(CHECKSUM(NEWID()))%2+65)
                     + CHAR(ABS(CHECKSUM(NEWID()))%2+65)
                     + CHAR(ABS(CHECKSUM(NEWID()))%2+65),
        Amount       = CAST(ABS(CHECKSUM(NEWID()))%1000000/100.0 AS MONEY),
        Quantity     = ABS(CHECKSUM(NEWID()))%50000+1,
        Date         = CAST(RAND(CHECKSUM(NEWID()))*3653.0+36524.0 AS DATETIME),
        Year         = CAST(NULL AS SMALLINT),
        Quarter      = CAST(NULL AS TINYINT)
   INTO #SomeTable3
   FROM Master.sys.SysColumns t1
  CROSS JOIN
        Master.sys.SysColumns t2 

--===== Fill in the Year and Quarter columns from the Date column
 UPDATE #SomeTable3
    SET Year    = DATEPART(yy,Date),
        Quarter = DATEPART(qq,Date)

--===== A table is not properly formed unless a Primary Key has been assigned
     -- Takes about 1 second to execute.
  ALTER TABLE #SomeTable3
        ADD PRIMARY KEY CLUSTERED (RowNum)
CREATE NONCLUSTERED INDEX IX_#SomeTable3_Cover1 
    ON dbo.#SomeTable3 (Company, Year)
       INCLUDE (Amount, Quantity, Quarter) 

create statistics scompanyyear on #sometable3(company, year) with fullscan, norecompute;
GO
insert into #timer values ('Finished Loading Test Data',getdate());
go
--=====--
--===== "Normal" Pivot
 SELECT amt.Company,
        amt.Year,
        COALESCE(amt.[1],0) AS Q1Amt,
        COALESCE(qty.[1],0) AS Q1Qty,
        COALESCE(amt.[2],0) AS Q2Amt,
        COALESCE(qty.[2],0) AS Q2Qty,
        COALESCE(amt.[3],0) AS Q3Amt,
        COALESCE(qty.[3],0) AS Q3Qty,
        COALESCE(amt.[4],0) AS Q4Amt,
        COALESCE(qty.[4],0) AS Q5Qty,
        COALESCE(amt.[1],0)+COALESCE(amt.[2],0)+COALESCE(amt.[3],0)+COALESCE(amt.[4],0) AS TotalAmt,
        COALESCE(qty.[1],0)+COALESCE(qty.[2],0)+COALESCE(qty.[3],0)+COALESCE(qty.[4],0) AS TotalQty
   into #NormalPivot_prep
   FROM (SELECT Company, Year, Quarter, Amount FROM #SomeTable3) t1
        PIVOT (SUM(Amount) FOR Quarter IN ([1], [2], [3], [4])) AS amt
  INNER JOIN 
        (SELECT Company, Year, Quarter, Quantity FROM #SomeTable3) t2
        PIVOT (SUM(Quantity) FOR Quarter IN ([1], [2], [3], [4])) AS qty
     ON qty.Company = amt.Company 
    AND qty.Year    = amt.Year         
  ORDER BY amt.Company, amt.Year
go
--insert into #timer values ('Finished Normal Pivot',getdate());
go
--=====--
--===== "Normal" Cross Tab
 SELECT Company,
        Year,
        SUM(CASE WHEN Quarter = 1 THEN Amount   ELSE 0 END) AS Q1Amt,
        SUM(CASE WHEN Quarter = 1 THEN Quantity ELSE 0 END) AS Q1Qty,
        SUM(CASE WHEN Quarter = 2 THEN Amount   ELSE 0 END) AS Q2Amt,
        SUM(CASE WHEN Quarter = 2 THEN Quantity ELSE 0 END) AS Q2Qty,
        SUM(CASE WHEN Quarter = 3 THEN Amount   ELSE 0 END) AS Q3Amt,
        SUM(CASE WHEN Quarter = 3 THEN Quantity ELSE 0 END) AS Q3Qty,
        SUM(CASE WHEN Quarter = 4 THEN Amount   ELSE 0 END) AS Q4Amt,
        SUM(CASE WHEN Quarter = 4 THEN Quantity ELSE 0 END) AS Q4Qty,
        SUM(Amount)   AS TotalAmt,
        SUM(Quantity) AS TotalQty
   into #NormalCrossTab_prep
   FROM #SomeTable3
  GROUP BY Company, Year
  ORDER BY Company, Year
go
--insert into #timer values ('Finished Normal Cross Tab',getdate());
insert into #timer values ('Finished Prep',getdate());
go
--=====--
--===== "Normal" Pivot
 SELECT amt.Company,
        amt.Year,
        COALESCE(amt.[1],0) AS Q1Amt,
        COALESCE(qty.[1],0) AS Q1Qty,
        COALESCE(amt.[2],0) AS Q2Amt,
        COALESCE(qty.[2],0) AS Q2Qty,
        COALESCE(amt.[3],0) AS Q3Amt,
        COALESCE(qty.[3],0) AS Q3Qty,
        COALESCE(amt.[4],0) AS Q4Amt,
        COALESCE(qty.[4],0) AS Q5Qty,
        COALESCE(amt.[1],0)+COALESCE(amt.[2],0)+COALESCE(amt.[3],0)+COALESCE(amt.[4],0) AS TotalAmt,
        COALESCE(qty.[1],0)+COALESCE(qty.[2],0)+COALESCE(qty.[3],0)+COALESCE(qty.[4],0) AS TotalQty
   into #NormalPivot
   FROM (SELECT Company, Year, Quarter, Amount FROM #SomeTable3) t1
        PIVOT (SUM(Amount) FOR Quarter IN ([1], [2], [3], [4])) AS amt
  INNER JOIN 
        (SELECT Company, Year, Quarter, Quantity FROM #SomeTable3) t2
        PIVOT (SUM(Quantity) FOR Quarter IN ([1], [2], [3], [4])) AS qty
     ON qty.Company = amt.Company 
    AND qty.Year    = amt.Year         
  ORDER BY amt.Company, amt.Year
go
insert into #timer values ('Finished Normal Pivot',getdate());
go
--=====--
--===== "Normal" Cross Tab
 SELECT Company,
        Year,
        SUM(CASE WHEN Quarter = 1 THEN Amount   ELSE 0 END) AS Q1Amt,
        SUM(CASE WHEN Quarter = 1 THEN Quantity ELSE 0 END) AS Q1Qty,
        SUM(CASE WHEN Quarter = 2 THEN Amount   ELSE 0 END) AS Q2Amt,
        SUM(CASE WHEN Quarter = 2 THEN Quantity ELSE 0 END) AS Q2Qty,
        SUM(CASE WHEN Quarter = 3 THEN Amount   ELSE 0 END) AS Q3Amt,
        SUM(CASE WHEN Quarter = 3 THEN Quantity ELSE 0 END) AS Q3Qty,
        SUM(CASE WHEN Quarter = 4 THEN Amount   ELSE 0 END) AS Q4Amt,
        SUM(CASE WHEN Quarter = 4 THEN Quantity ELSE 0 END) AS Q4Qty,
        SUM(Amount)   AS TotalAmt,
        SUM(Quantity) AS TotalQty
   into #NormalCrossTab
   FROM #SomeTable3
  GROUP BY Company, Year
  ORDER BY Company, Year
go
insert into #timer values ('Finished Normal Cross Tab',getdate());
go
--=====--
select 
    o.what
  , started=isnull(convert(varchar(30),x.ended),o.ended)
  , ended=convert(varchar(30),o.ended)
  , DurationInMs=datediff(millisecond,x.ended,o.ended)
from #timer o
  outer apply (select top 1 ended from #timer i where i.ended < o.ended order by i.ended desc) as x

returns:

+----------------------------+---------------------+---------------------+--------------+
|            what            |       started       |        ended        | DurationInMs |
+----------------------------+---------------------+---------------------+--------------+
| Start                      | Feb 19 2017  7:19PM | Feb 19 2017  7:19PM | NULL         |
| Finished Loading Test Data | Feb 19 2017  7:19PM | Feb 19 2017  7:19PM | 5260         |
| Finished Prep              | Feb 19 2017  7:19PM | Feb 19 2017  7:19PM | 1003         |
| Finished Normal Pivot      | Feb 19 2017  7:19PM | Feb 19 2017  7:19PM | 550          |
| Finished Normal Cross Tab  | Feb 19 2017  7:19PM | Feb 19 2017  7:19PM | 513          |
+----------------------------+---------------------+---------------------+--------------+

"Pre-aggregated"

rextester: http://rextester.com/WBGUYR51251

create table #timer (what varchar(64), ended datetime);
insert into #timer values ('Start',getdate());
go
SELECT TOP 300000 --<<Look!  Change this number for testing different size tables
        RowNum       = IDENTITY(INT,1,1),
        Company      = CHAR(ABS(CHECKSUM(NEWID()))%2+65)
                     + CHAR(ABS(CHECKSUM(NEWID()))%2+65)
                     + CHAR(ABS(CHECKSUM(NEWID()))%2+65),
        Amount       = CAST(ABS(CHECKSUM(NEWID()))%1000000/100.0 AS MONEY),
        Quantity     = ABS(CHECKSUM(NEWID()))%50000+1,
        Date         = CAST(RAND(CHECKSUM(NEWID()))*3653.0+36524.0 AS DATETIME),
        Year         = CAST(NULL AS SMALLINT),
        Quarter      = CAST(NULL AS TINYINT)
   INTO #SomeTable3
   FROM Master.sys.SysColumns t1
  CROSS JOIN
        Master.sys.SysColumns t2 

--===== Fill in the Year and Quarter columns from the Date column
 UPDATE #SomeTable3
    SET Year    = DATEPART(yy,Date),
        Quarter = DATEPART(qq,Date)

--===== A table is not properly formed unless a Primary Key has been assigned
     -- Takes about 1 second to execute.
  ALTER TABLE #SomeTable3
        ADD PRIMARY KEY CLUSTERED (RowNum)
CREATE NONCLUSTERED INDEX IX_#SomeTable3_Cover1 
    ON dbo.#SomeTable3 (Company, Year)
       INCLUDE (Amount, Quantity, Quarter) 

create statistics scompanyyear on #sometable3(company, year) with fullscan, norecompute;
GO
insert into #timer values ('Finished Loading Test Data',getdate());
go
--=====--
--===== "Pre-aggregated" Pivot
SELECT amt.Company,
        amt.Year,
        COALESCE(amt.[1],0) AS Q1Amt,
        COALESCE(qty.[1],0) AS Q1Qty,
        COALESCE(amt.[2],0) AS Q2Amt,
        COALESCE(qty.[2],0) AS Q2Qty,
        COALESCE(amt.[3],0) AS Q3Amt,
        COALESCE(qty.[3],0) AS Q3Qty,
        COALESCE(amt.[4],0) AS Q4Amt,
        COALESCE(qty.[4],0) AS Q5Qty,
        COALESCE(amt.[1],0)+COALESCE(amt.[2],0)+COALESCE(amt.[3],0)+COALESCE(amt.[4],0) AS TotalAmt,
        COALESCE(qty.[1],0)+COALESCE(qty.[2],0)+COALESCE(qty.[3],0)+COALESCE(qty.[4],0) AS TotalQty
   into #preA_Pivot_prep
   FROM (SELECT Company, Year, Quarter, SUM(Amount) AS Amount FROM #SomeTable3 GROUP BY Company, Year, Quarter) t1
        PIVOT (SUM(Amount) FOR Quarter IN ([1], [2], [3], [4])) AS amt
  INNER JOIN 
        (SELECT Company, Year, Quarter, SUM(Quantity) AS Quantity FROM #SomeTable3 GROUP BY Company, Year, Quarter) t2
        PIVOT (SUM(Quantity) FOR Quarter IN ([1], [2], [3], [4])) AS qty
     ON qty.Company = amt.Company 
    AND qty.Year    = amt.Year         
  ORDER BY amt.Company, amt.Year
go
--insert into #timer values ('Finished "Pre-aggregated" Pivot',getdate());
go
--=====--
--===== "Pre-aggregated" Cross Tab
SELECT Company,
        Year,
        SUM(CASE WHEN Quarter = 1 THEN Amount   ELSE 0 END) AS Q1Amt,
        SUM(CASE WHEN Quarter = 1 THEN Quantity ELSE 0 END) AS Q1Qty,
        SUM(CASE WHEN Quarter = 2 THEN Amount   ELSE 0 END) AS Q2Amt,
        SUM(CASE WHEN Quarter = 2 THEN Quantity ELSE 0 END) AS Q2Qty,
        SUM(CASE WHEN Quarter = 3 THEN Amount   ELSE 0 END) AS Q3Amt,
        SUM(CASE WHEN Quarter = 3 THEN Quantity ELSE 0 END) AS Q3Qty,
        SUM(CASE WHEN Quarter = 4 THEN Amount   ELSE 0 END) AS Q4Amt,
        SUM(CASE WHEN Quarter = 4 THEN Quantity ELSE 0 END) AS Q4Qty,
        SUM(Amount)   AS TotalAmt,
        SUM(Quantity) AS TotalQty
   into #preA_CrossTab_prep
   FROM (SELECT Company,Year,Quarter,SUM(Amount) AS Amount,SUM(Quantity) AS Quantity
           FROM #SomeTable3 GROUP BY Company,Year,Quarter) d
  GROUP BY Company, Year
  ORDER BY Company, Year
go
--insert into #timer values ('Finished "Pre-aggregated" Cross Tab',getdate());
go
--=====--
insert into #timer values ('Finished Prep',getdate());
--=====--
--===== "Pre-aggregated" Pivot
SELECT amt.Company,
        amt.Year,
        COALESCE(amt.[1],0) AS Q1Amt,
        COALESCE(qty.[1],0) AS Q1Qty,
        COALESCE(amt.[2],0) AS Q2Amt,
        COALESCE(qty.[2],0) AS Q2Qty,
        COALESCE(amt.[3],0) AS Q3Amt,
        COALESCE(qty.[3],0) AS Q3Qty,
        COALESCE(amt.[4],0) AS Q4Amt,
        COALESCE(qty.[4],0) AS Q5Qty,
        COALESCE(amt.[1],0)+COALESCE(amt.[2],0)+COALESCE(amt.[3],0)+COALESCE(amt.[4],0) AS TotalAmt,
        COALESCE(qty.[1],0)+COALESCE(qty.[2],0)+COALESCE(qty.[3],0)+COALESCE(qty.[4],0) AS TotalQty
   into #preA_Pivot
   FROM (SELECT Company, Year, Quarter, SUM(Amount) AS Amount FROM #SomeTable3 GROUP BY Company, Year, Quarter) t1
        PIVOT (SUM(Amount) FOR Quarter IN ([1], [2], [3], [4])) AS amt
  INNER JOIN 
        (SELECT Company, Year, Quarter, SUM(Quantity) AS Quantity FROM #SomeTable3 GROUP BY Company, Year, Quarter) t2
        PIVOT (SUM(Quantity) FOR Quarter IN ([1], [2], [3], [4])) AS qty
     ON qty.Company = amt.Company 
    AND qty.Year    = amt.Year         
  ORDER BY amt.Company, amt.Year
go
insert into #timer values ('Finished "Pre-aggregated" Pivot',getdate());
go
--=====--
--===== "Pre-aggregated" Cross Tab
SELECT Company,
        Year,
        SUM(CASE WHEN Quarter = 1 THEN Amount   ELSE 0 END) AS Q1Amt,
        SUM(CASE WHEN Quarter = 1 THEN Quantity ELSE 0 END) AS Q1Qty,
        SUM(CASE WHEN Quarter = 2 THEN Amount   ELSE 0 END) AS Q2Amt,
        SUM(CASE WHEN Quarter = 2 THEN Quantity ELSE 0 END) AS Q2Qty,
        SUM(CASE WHEN Quarter = 3 THEN Amount   ELSE 0 END) AS Q3Amt,
        SUM(CASE WHEN Quarter = 3 THEN Quantity ELSE 0 END) AS Q3Qty,
        SUM(CASE WHEN Quarter = 4 THEN Amount   ELSE 0 END) AS Q4Amt,
        SUM(CASE WHEN Quarter = 4 THEN Quantity ELSE 0 END) AS Q4Qty,
        SUM(Amount)   AS TotalAmt,
        SUM(Quantity) AS TotalQty
   into #preA_CrossTab
   FROM (SELECT Company,Year,Quarter,SUM(Amount) AS Amount,SUM(Quantity) AS Quantity
           FROM #SomeTable3 GROUP BY Company,Year,Quarter) d
  GROUP BY Company, Year
  ORDER BY Company, Year
go
insert into #timer values ('Finished "Pre-aggregated" Cross Tab',getdate());
go
--=====--
select 
    o.what
  , started=isnull(convert(varchar(30),x.ended),o.ended)
  , ended=convert(varchar(30),o.ended)
  , DurationInMs=datediff(millisecond,x.ended,o.ended)
from #timer o
  outer apply (select top 1 ended from #timer i where i.ended < o.ended order by i.ended desc) as x

returns:

+-------------------------------------+---------------------+---------------------+--------------+
|                what                 |       started       |        ended        | DurationInMs |
+-------------------------------------+---------------------+---------------------+--------------+
| Start                               | Feb 19 2017  7:23PM | Feb 19 2017  7:23PM | NULL         |
| Finished Loading Test Data          | Feb 19 2017  7:23PM | Feb 19 2017  7:23PM | 5440         |
| Finished Prep                       | Feb 19 2017  7:23PM | Feb 19 2017  7:23PM | 1513         |
| Finished "Pre-aggregated" Pivot     | Feb 19 2017  7:23PM | Feb 19 2017  7:23PM | 683          |
| Finished "Pre-aggregated" Cross Tab | Feb 19 2017  7:23PM | Feb 19 2017  7:23PM | 370          |
+-------------------------------------+---------------------+---------------------+--------------+

"Pre-aggregated" with CTE

rextester: http://rextester.com/WCTJH5484

create table #timer (what varchar(64), ended datetime);
insert into #timer values ('Start',getdate());
go

SELECT TOP 300000 --<<Look!  Change this number for testing different size tables
        RowNum       = IDENTITY(INT,1,1),
        Company      = CHAR(ABS(CHECKSUM(NEWID()))%2+65)
                     + CHAR(ABS(CHECKSUM(NEWID()))%2+65)
                     + CHAR(ABS(CHECKSUM(NEWID()))%2+65),
        Amount       = CAST(ABS(CHECKSUM(NEWID()))%1000000/100.0 AS MONEY),
        Quantity     = ABS(CHECKSUM(NEWID()))%50000+1,
        Date         = CAST(RAND(CHECKSUM(NEWID()))*3653.0+36524.0 AS DATETIME),
        Year         = CAST(NULL AS SMALLINT),
        Quarter      = CAST(NULL AS TINYINT)
   INTO #SomeTable3
   FROM Master.sys.SysColumns t1
  CROSS JOIN
        Master.sys.SysColumns t2 

--===== Fill in the Year and Quarter columns from the Date column
 UPDATE #SomeTable3
    SET Year    = DATEPART(yy,Date),
        Quarter = DATEPART(qq,Date)

--===== A table is not properly formed unless a Primary Key has been assigned
     -- Takes about 1 second to execute.
  ALTER TABLE #SomeTable3
        ADD PRIMARY KEY CLUSTERED (RowNum)
CREATE NONCLUSTERED INDEX IX_#SomeTable3_Cover1 
    ON dbo.#SomeTable3 (Company, Year)
       INCLUDE (Amount, Quantity, Quarter) 

create statistics syearquarter on #sometable3(year,quarter) with fullscan, norecompute;
GO
insert into #timer values ('Finished Loading Test Data',getdate());
go
--=====--
--===== "Pre-aggregated" Pivot with CTE
;WITH
ctePreAgg AS
(SELECT Company,Year,Quarter,SUM(Amount) AS Amount,SUM(Quantity) AS Quantity
   FROM #SomeTable3 
  GROUP BY Company,Year,Quarter
)
 SELECT amt.Company,
        amt.Year,
        COALESCE(amt.[1],0) AS Q1Amt,
        COALESCE(qty.[1],0) AS Q1Qty,
        COALESCE(amt.[2],0) AS Q2Amt,
        COALESCE(qty.[2],0) AS Q2Qty,
        COALESCE(amt.[3],0) AS Q3Amt,
        COALESCE(qty.[3],0) AS Q3Qty,
        COALESCE(amt.[4],0) AS Q4Amt,
        COALESCE(qty.[4],0) AS Q5Qty,
        COALESCE(amt.[1],0)+COALESCE(amt.[2],0)+COALESCE(amt.[3],0)+COALESCE(amt.[4],0) AS TotalAmt,
        COALESCE(qty.[1],0)+COALESCE(qty.[2],0)+COALESCE(qty.[3],0)+COALESCE(qty.[4],0) AS TotalQty
   into #prea_Pivot_wcte_prep
   FROM (SELECT Company, Year, Quarter, Amount FROM ctePreAgg) AS t1
        PIVOT (SUM(Amount) FOR Quarter IN ([1], [2], [3], [4])) AS amt
  INNER JOIN 
        (SELECT Company, Year, Quarter, Quantity FROM ctePreAgg) AS t2
        PIVOT (SUM(Quantity) FOR Quarter IN ([1], [2], [3], [4])) AS qty
     ON qty.Company = amt.Company 
    AND qty.Year    = amt.Year         
  ORDER BY amt.Company, amt.Year
go
--insert into #timer values ('Finished "Pre-aggregated" Pivot with CTE',getdate());
go
--=====--
--===== "Pre-aggregated" Cross Tab with CTE
;WITH
ctePreAgg AS
(SELECT Company,Year,Quarter,SUM(Amount) AS Amount,SUM(Quantity) AS Quantity
   FROM #SomeTable3 
  GROUP BY Company,Year,Quarter
)
 SELECT Company,
        Year,
        SUM(CASE WHEN Quarter = 1 THEN Amount   ELSE 0 END) AS Q1Amt,
        SUM(CASE WHEN Quarter = 1 THEN Quantity ELSE 0 END) AS Q1Qty,
        SUM(CASE WHEN Quarter = 2 THEN Amount   ELSE 0 END) AS Q2Amt,
        SUM(CASE WHEN Quarter = 2 THEN Quantity ELSE 0 END) AS Q2Qty,
        SUM(CASE WHEN Quarter = 3 THEN Amount   ELSE 0 END) AS Q3Amt,
        SUM(CASE WHEN Quarter = 3 THEN Quantity ELSE 0 END) AS Q3Qty,
        SUM(CASE WHEN Quarter = 4 THEN Amount   ELSE 0 END) AS Q4Amt,
        SUM(CASE WHEN Quarter = 4 THEN Quantity ELSE 0 END) AS Q4Qty,
        SUM(Amount)   AS TotalAmt,
        SUM(Quantity) AS TotalQty
   into #prea_CrossTab_wcte_prep
   FROM ctePreAgg
  GROUP BY Company, Year
  ORDER BY Company, Year
go
--insert into #timer values ('Finished "Pre-aggregated" Cross Tab with CTE',getdate());
go
--=====--
insert into #timer values ('Finished Prep',getdate());
go
--=====--
--===== "Pre-aggregated" Pivot with CTE
;WITH
ctePreAgg AS
(SELECT Company,Year,Quarter,SUM(Amount) AS Amount,SUM(Quantity) AS Quantity
   FROM #SomeTable3 
  GROUP BY Company,Year,Quarter
)
 SELECT amt.Company,
        amt.Year,
        COALESCE(amt.[1],0) AS Q1Amt,
        COALESCE(qty.[1],0) AS Q1Qty,
        COALESCE(amt.[2],0) AS Q2Amt,
        COALESCE(qty.[2],0) AS Q2Qty,
        COALESCE(amt.[3],0) AS Q3Amt,
        COALESCE(qty.[3],0) AS Q3Qty,
        COALESCE(amt.[4],0) AS Q4Amt,
        COALESCE(qty.[4],0) AS Q5Qty,
        COALESCE(amt.[1],0)+COALESCE(amt.[2],0)+COALESCE(amt.[3],0)+COALESCE(amt.[4],0) AS TotalAmt,
        COALESCE(qty.[1],0)+COALESCE(qty.[2],0)+COALESCE(qty.[3],0)+COALESCE(qty.[4],0) AS TotalQty
   into #prea_Pivot_wcte
   FROM (SELECT Company, Year, Quarter, Amount FROM ctePreAgg) AS t1
        PIVOT (SUM(Amount) FOR Quarter IN ([1], [2], [3], [4])) AS amt
  INNER JOIN 
        (SELECT Company, Year, Quarter, Quantity FROM ctePreAgg) AS t2
        PIVOT (SUM(Quantity) FOR Quarter IN ([1], [2], [3], [4])) AS qty
     ON qty.Company = amt.Company 
    AND qty.Year    = amt.Year         
  ORDER BY amt.Company, amt.Year
go
insert into #timer values ('Finished "Pre-aggregated" Pivot with CTE',getdate());
go
--=====--
--===== "Pre-aggregated" Cross Tab with CTE
;WITH
ctePreAgg AS
(SELECT Company,Year,Quarter,SUM(Amount) AS Amount,SUM(Quantity) AS Quantity
   FROM #SomeTable3 
  GROUP BY Company,Year,Quarter
)
 SELECT Company,
        Year,
        SUM(CASE WHEN Quarter = 1 THEN Amount   ELSE 0 END) AS Q1Amt,
        SUM(CASE WHEN Quarter = 1 THEN Quantity ELSE 0 END) AS Q1Qty,
        SUM(CASE WHEN Quarter = 2 THEN Amount   ELSE 0 END) AS Q2Amt,
        SUM(CASE WHEN Quarter = 2 THEN Quantity ELSE 0 END) AS Q2Qty,
        SUM(CASE WHEN Quarter = 3 THEN Amount   ELSE 0 END) AS Q3Amt,
        SUM(CASE WHEN Quarter = 3 THEN Quantity ELSE 0 END) AS Q3Qty,
        SUM(CASE WHEN Quarter = 4 THEN Amount   ELSE 0 END) AS Q4Amt,
        SUM(CASE WHEN Quarter = 4 THEN Quantity ELSE 0 END) AS Q4Qty,
        SUM(Amount)   AS TotalAmt,
        SUM(Quantity) AS TotalQty
   into #prea_CrossTab_wcte
   FROM ctePreAgg
  GROUP BY Company, Year
  ORDER BY Company, Year
go
insert into #timer values ('Finished "Pre-aggregated" Cross Tab with CTE',getdate());
go
--=====--
select 
    o.what
  , started=isnull(convert(varchar(30),x.ended),o.ended)
  , ended=convert(varchar(30),o.ended)
  , DurationInMs=datediff(millisecond,x.ended,o.ended)
from #timer o
  outer apply (select top 1 ended from #timer i where i.ended < o.ended order by i.ended desc) as x

returns:

+----------------------------------------------+---------------------+---------------------+--------------+
|                     what                     |       started       |        ended        | DurationInMs |
+----------------------------------------------+---------------------+---------------------+--------------+
| Start                                        | Feb 19 2017  7:25PM | Feb 19 2017  7:25PM | NULL         |
| Finished Loading Test Data                   | Feb 19 2017  7:25PM | Feb 19 2017  7:26PM | 5723         |
| Finished Prep                                | Feb 19 2017  7:26PM | Feb 19 2017  7:26PM | 950          |
| Finished "Pre-aggregated" Pivot with CTE     | Feb 19 2017  7:26PM | Feb 19 2017  7:26PM | 580          |
| Finished "Pre-aggregated" Cross Tab with CTE | Feb 19 2017  7:26PM | Feb 19 2017  7:26PM | 323          |
+----------------------------------------------+---------------------+---------------------+--------------+
  • I am a little skeptical about the article regarding it's purpose. Which appears to be to convince people it is not necessary to use PIVOT even after upgrading from 2000 to 2005. I couldn't immediately find a problem with the methodology, but, as MartinSmith says, there are many things that can potentially skew performance testing results. I once thought I had cut the time for a highly used SP in half once, but then found it was because in altering the SP it had generated a new execution plan, and recompiling the original would help it as well. In truth my improvement was only 10%. – Matthew Sontum Feb 19 '17 at 17:59
6

Just a note to correct a false assumption:

There are three, not two, main ways to write a pivot. The third is using a driving table and several joins (with either LEFT JOIN or OUTER/CROSS APPLY):

This may be more or less efficient, depending on several details (table distributions, indexes, etc) and requirements of the specific pivot operation. It has several differences with the SUM / GROUP BY method:

  • it can avoid scanning the whole table if there are appropriate indexes (appropriate meaning: different depending on the WHERE clause, the GROUP BY clause and the columns to be aggregated). In the specific example, an index on (RateItemTypeID, RateID) INCLUDE (UnitPrice) if _WherClause_ is empty.

    • This can be beneficial on several occasions:
      If the table for example has hundreds of different RateItemTypeID values but our query is only interested in a few. Scanning the whole table (or even a whole index) vs. seeking a smaller part of a narrower NCI, I'd expect the 2nd to be more efficient.
    • It can be counter-productive of course because it requires different indexes for different WHERE predicates and aggregated columns.
  • the GROUP BY and even the whole driving subquery can often be replaced by another table (a Rate table in the specific example).

  • on several pivot variations the GROUP BY in the subqueries can also be removed and the subqueries converted to simple LEFT joins (if for example there is a UNIQUE constraint on (RateID, RateItemTypeID) in the specific case). This shows that the SUM in the "SUM / GROUP BY" method is (in these cases) there only because of the GROUP BY and summing one value (and several Nulls).

The query:

SELECT 
    d.RateID,
    Sum1 = COALESCE(s1.Sum1, 0),
    Sum2 = COALESCE(s2.Sum2, 0),
    Sum3 = COALESCE(s3.Sum3, 0)  
FROM 
    ( SELECT RateID    -- 
      FROM rate_item 
      WHERE _WhereClause_
      GROUP BY RateID
    ) AS d             -- driving table with the DISTINCT RateID values
  OUTER APPLY
    ( SELECT Sum1 = SUM(r1.UnitPrice)
      FROM rate_item AS r1
      WHERE _WhereClause_
        AND r1.RateItemTypeID = 1 
        AND r1.RateID = d.RateID
    ) AS s1
  OUTER APPLY
    ( SELECT Sum2 = SUM(r2.UnitPrice)
      FROM rate_item AS r2
      WHERE _WhereClause_
        AND r2.RateItemTypeID = 2 
        AND r2.RateID = d.RateID
    ) AS s2
  OUTER APPLY
    ( SELECT Sum3 = SUM(r3.UnitPrice)
      FROM rate_item AS r3
      WHERE _WhereClause_
        AND r3.RateItemTypeID = 3 
        AND r3.RateID = d.RateID
    ) AS s3 ;
5

SQL Server transforms below query:

SELECT 
RateID, [1], [2], [3]
FROM PertinentRates
PIVOT (SUM(UnitPrice) FOR RateItemTypeID IN ([1], [2], [3])) PVT)

to:

SELECT 
RateID
SUM(CASE WHEN RateItemTypeID = 1 THEN UnitPrice ELSE 0 END),
SUM(CASE WHEN RateItemTypeID = 2 THEN UnitPrice ELSE 0 END),
SUM(CASE WHEN RateItemTypeID = 3 THEN UnitPrice ELSE 0 END)
 FROM rate_item WHERE supplierid = 2882874 AND rateplanid = 1 AND rateitemtypeid IN (1, 2, 3)
          GROUP BY RateID

so choosing one over another ,AFAIK boils down to readability

Below is short demo:

CREATE TABLE #Sales (EmpId INT, Yr INT, Sales MONEY)
INSERT #Sales VALUES(1, 2005, 12000)
INSERT #Sales VALUES(1, 2006, 18000)
INSERT #Sales VALUES(1, 2007, 25000)
INSERT #Sales VALUES(2, 2005, 15000)
INSERT #Sales VALUES(2, 2006, 6000)
INSERT #Sales VALUES(3, 2006, 20000)
INSERT #Sales VALUES(3, 2007, 24000)

now the queries

SELECT EmpId, [2005], [2006], [2007]
FROM (SELECT EmpId, Yr, Sales FROM #Sales) AS s
PIVOT (SUM(Sales) FOR Yr IN ([2005], [2006], [2007])) AS p    

select 
empid,
sum(Case when yr=2005 then sales end) '2005',
sum(Case when yr=2006 then sales end) '2006',
sum(Case when yr=2007 then sales end) '2007'
from
#sales
group by empid

now when both queries are executed in batch, both occupy same cost and the plans are much the same:

enter image description here

the SHOWPLAN_TEXT is

Pivot

  |--Compute Scalar(DEFINE:([Expr1003]=CASE WHEN [Expr1018]=(0) THEN NULL ELSE [Expr1019] END, [Expr1004]=CASE WHEN [Expr1020]=(0) THEN NULL ELSE [Expr1021] END, [Expr1005]=CASE WHEN [Expr1022]=(0) THEN NULL ELSE [Expr1023] END))
       |--Stream Aggregate(GROUP BY:([tempdb].[dbo].[#Sales].[EmpId]) DEFINE:([Expr1018]=COUNT_BIG(CASE WHEN [tempdb].[dbo].[#Sales].[Yr]=(2005) THEN [tempdb].[dbo].[#Sales].[Sales] ELSE NULL END), [Expr1019]=SUM(CASE WHEN [tempdb].[dbo].[#Sales].[Yr]=(2005) THEN [tempdb].[dbo].[#Sales].[Sales] ELSE NULL END), [Expr1020]=COUNT_BIG(CASE WHEN [tempdb].[dbo].[#Sales].[Yr]=(2006) THEN [tempdb].[dbo].[#Sales].[Sales] ELSE NULL END), [Expr1021]=SUM(CASE WHEN [tempdb].[dbo].[#Sales].[Yr]=(2006) THEN [tempdb].[dbo].[#Sales].[Sales] ELSE NULL END), [Expr1022]=COUNT_BIG(CASE WHEN [tempdb].[dbo].[#Sales].[Yr]=(2007) THEN [tempdb].[dbo].[#Sales].[Sales] ELSE NULL END), [Expr1023]=SUM(CASE WHEN [tempdb].[dbo].[#Sales].[Yr]=(2007) THEN [tempdb].[dbo].[#Sales].[Sales] ELSE NULL END)))
            |--Sort(ORDER BY:([tempdb].[dbo].[#Sales].[EmpId] ASC))
                 |--Table Scan(OBJECT:([tempdb].[dbo].[#Sales]))

CASE

  |--Compute Scalar(DEFINE:([Expr1003]=CASE WHEN [Expr1021]=(0) THEN NULL ELSE [Expr1022] END, [Expr1004]=CASE WHEN [Expr1023]=(0) THEN NULL ELSE [Expr1024] END, [Expr1005]=CASE WHEN [Expr1025]=(0) THEN NULL ELSE [Expr1026] END))
       |--Stream Aggregate(GROUP BY:([tempdb].[dbo].[#sales].[EmpId]) DEFINE:([Expr1021]=COUNT_BIG([Expr1006]), [Expr1022]=SUM([Expr1006]), [Expr1023]=COUNT_BIG([Expr1007]), [Expr1024]=SUM([Expr1007]), [Expr1025]=COUNT_BIG([Expr1008]), [Expr1026]=SUM([Expr1008])))
            |--Compute Scalar(DEFINE:([Expr1006]=CASE WHEN [tempdb].[dbo].[#sales].[Yr]=(2005) THEN [tempdb].[dbo].[#sales].[Sales] ELSE NULL END, [Expr1007]=CASE WHEN [tempdb].[dbo].[#sales].[Yr]=(2006) THEN [tempdb].[dbo].[#sales].[Sales] ELSE NULL END, [Expr1008]=CASE WHEN [tempdb].[dbo].[#sales].[Yr]=(2007) THEN [tempdb].[dbo].[#sales].[Sales] ELSE NULL END))
                 |--Sort(ORDER BY:([tempdb].[dbo].[#sales].[EmpId] ASC))
                      |--Table Scan(OBJECT:([tempdb].[dbo].[#sales]))
  • @TheGameiswar there is a difference between the first 2 queries in your answer (supplierid = 2882874 AND rateplanid = 1) – ypercubeᵀᴹ Feb 19 '17 at 20:03

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.