4

I have this query running around 15 seconds on my system. The query plan is here. I believe the main problem is around here, where the optimizer chooses nested loops and is mistaken with cardinality:

enter image description here

Tables _Reference65215 and _InfoRg100966 have 454 and 3332 rows respectively. If I force SQL Server to use hash match here the execution time drops to 2 seconds.

I cannot use any kind of hints here as the code comes from the platform. The platform uses its own language which then gets interpreted to T-SQL. I can change the query. Stats are up to date. Maxdop 1 is a general recommendation from the vendor of the platform

What might be the reason of choosing nested loops and how to fix it? Maybe there is something else that is worth attention?

  EXEC sp_executesql 
         N'SELECT TOP 45 T1._IDRRef, 
                  T1._Marked, 
                  T1._Number, 
                  T1._Date_Time, 
                  T1._Posted, 
                  T1._Fld74271RRef,
                  CASE
                    WHEN T1._Fld74272RRef=@P1
                    THEN T1._Fld74307RRef
                    ELSE T1._Fld74272RRef
                  END, 
                  T1._Fld74273, 
                  T1._Fld74274RRef, 
                  T1._Fld74281,
                  CASE
                    WHEN T1._Fld74272RRef=@P2
                    THEN 0x000100DD
                    WHEN T1._Fld74272RRef=@P3
                         OR T1._Fld74272RRef=@P4
                         OR T1._Fld74272RRef=@P5
                    THEN 0x0000FF7E
                    ELSE 0x0000FF3E
                  END,
                  CASE
                    WHEN T1._Fld74272RRef=@P6
                    THEN T1._Fld74284RRef
                    WHEN T1._Fld74272RRef=@P7
                         OR T1._Fld74272RRef=@P8
                         OR T1._Fld74272RRef=@P9
                    THEN T1._Fld74285RRef
                    ELSE T1._Fld74282RRef
                  END, 
                  T1._Fld74289RRef, 
                  T1._Fld74303,
                  CASE
                    WHEN T1._Fld74333RRef IN( SELECT TOP 1 T12._IDRRef
                                              AS Q_001_F_000RRef
                                              FROM dbo._Reference65527 T12
                                              WHERE T12._Fld67386=@P10
                                              ORDER BY T12._Fld124008 )
                    THEN @P11
                    WHEN T1._Fld74333RRef IN( SELECT TOP 1 T13._IDRRef
                                              AS Q_002_F_000RRef
                                              FROM dbo._Reference65527 T13
                                              WHERE T13._Fld67386=@P12
                                              ORDER BY T13._Fld124008 DESC )
                    THEN @P13
                    ELSE @P14
                  END,
                  CASE
                    WHEN ISNULL( CAST( T2.Fld109931Balance_ AS NUMERIC(27, 2) ), @P15 )<=@P16
                         AND T1._Posted=0x01
                         AND T1._Fld74271RRef<>@P17
                    THEN 0x01
                    ELSE 0x00
                  END,
                  CASE
                    WHEN T11._Fld101929 IS NULL
                    THEN @P18
                    WHEN T11._Fld101929=0x01
                    THEN @P19
                    ELSE @P20
                  END
    FROM dbo._Document65911 T1
    LEFT OUTER JOIN
    (
      SELECT T3._Fld109927_TYPE
      AS Fld109927_TYPE, 
             T3._Fld109927_RTRef
      AS Fld109927_RTRef, 
             T3._Fld109927_RRRef
      AS Fld109927_RRRef, 
             CAST( SUM( T3._Fld109931 ) AS NUMERIC(27, 2) )
      AS Fld109931Balance_
      FROM dbo._AccumRgT109935 T3
      WHERE T3._Fld67386=@P21
            AND EXISTS
      (
        SELECT 0x01
        AS Q_001_F_000_
        FROM dbo._Reference65291 T4
        INNER JOIN
        dbo._Reference65215 T5
        ON T4._Fld118298=@P22
           AND EXISTS
        (
          SELECT 0x01
          AS Q_004_F_000_
          FROM dbo._InfoRg107349 T6
          WHERE T6._Fld67386=@P23
                AND T6._Fld107350RRef=T4._IDRRef
                    AND T6._Fld107351RRef=T5._IDRRef
        )
           AND T5._IDRRef IN
        (
          SELECT T7._Reference65215_IDRRef
          AS Q_005_F_000RRef
          FROM dbo._Reference65215_VT116326 T7
          INNER JOIN
          dbo._InfoRg106545 T8
          ON T8._Fld106547_TYPE=0x08
             AND T8._Fld106547_RTRef=0x0000FFD5
             AND T8._Fld106547_RRRef=@P24
             AND T8._Fld106546_TYPE=T7._Fld116328_TYPE
                 AND T8._Fld106546_RTRef=T7._Fld116328_RTRef
                 AND T8._Fld106546_RRRef=T7._Fld116328_RRRef
          WHERE T7._Fld67386=@P25
                AND T8._Fld67386=@P26
        )
        WHERE T5._Fld67386=@P27
              AND 0x08<>0x01
                  AND CASE
                        WHEN EXISTS
        (
          SELECT 0x01
          AS Q_002_F_000_
          FROM dbo._InfoRg100966 T9
          WHERE T9._Fld67386=@P28
                AND T9._Fld100967RRef=T5._IDRRef
                    AND T9._Fld100968_TYPE=0x08
                        AND T9._Fld100968_RTRef=0x0000FF7E
                        AND T9._Fld100968_RRRef=T3._Fld109930RRef
        )
                        THEN 0x01
                        ELSE 0x00
                      END=CASE
                            WHEN EXISTS
        (
          SELECT 0x01
          AS Q_003_F_000_
          FROM dbo._InfoRg100973 T10
          WHERE T10._Fld67386=@P29
                AND T10._Fld100974RRef=T5._IDRRef
                    AND T10._Fld100975_TYPE=0x08
                    AND T10._Fld100975_RTRef=0x0000FF7E
                    AND T10._Fld100976=0x00
        )
                            THEN 0x01
                            ELSE 0x00
                          END
      )
        AND T3._Period=@P30
            AND T3._Fld109931<>@P31
            AND T3._Fld109931<>@P32
      GROUP BY T3._Fld109927_TYPE, 
               T3._Fld109927_RTRef, 
               T3._Fld109927_RRRef
      HAVING CAST( SUM( T3._Fld109931 ) AS NUMERIC(27, 2) )<>0.0
    ) T2
    ON T2.Fld109927_TYPE=0x08
       AND T2.Fld109927_RTRef=0x00010177
       AND T2.Fld109927_RRRef=T1._IDRRef
        LEFT OUTER JOIN
        dbo._InfoRg101927 T11
        ON(0x08=T11._Fld101928_TYPE
           AND 0x00010177=T11._Fld101928_RTRef
           AND T1._IDRRef=T11._Fld101928_RRRef)
          AND T11._Fld67386=@P33
    WHERE T1._Fld67386=@P34
          AND EXISTS
    (
      SELECT 1
      FROM
      (
        SELECT 1
        AS SDBL_DUMMY
      ) SDBL_DUAL
      LEFT OUTER JOIN
      dbo._Document65911_VT74348 T14
      ON T1._IDRRef=T14._Document65911_IDRRef
         AND T14._Fld67386=T1._Fld67386
          LEFT OUTER JOIN
          dbo._Reference65239 T15
          ON T1._Fld74344_TYPE=0x08
             AND T1._Fld74344_RTRef=0x0000FED7
             AND T1._Fld74344_RRRef=T15._IDRRef
             AND T15._Fld67386=@P35
              LEFT OUTER JOIN
              dbo._Reference65757 T16
              ON T1._Fld74284RRef=T16._IDRRef
                 AND T16._Fld67386=@P36
      WHERE EXISTS
      (
        SELECT 0x01
        AS Q_001_F_000_
        FROM dbo._Reference65291 T17
        INNER JOIN
        dbo._Reference65215 T18
        ON T17._Fld118298=@P37
           AND EXISTS
        (
          SELECT 0x01
          AS Q_015_F_000_
          FROM dbo._InfoRg107349 T19
          WHERE T19._Fld67386=@P38
                AND T19._Fld107350RRef=T17._IDRRef
                    AND T19._Fld107351RRef=T18._IDRRef
        )
           AND T18._IDRRef IN
        (
          SELECT T20._Reference65215_IDRRef
          AS Q_016_F_000RRef
          FROM dbo._Reference65215_VT116326 T20
          INNER JOIN
          dbo._InfoRg106545 T21
          ON T21._Fld106547_TYPE=0x08
             AND T21._Fld106547_RTRef=0x0000FFD5
             AND T21._Fld106547_RRRef=@P39
             AND T21._Fld106546_TYPE=T20._Fld116328_TYPE
                 AND T21._Fld106546_RTRef=T20._Fld116328_RTRef
                 AND T21._Fld106546_RRRef=T20._Fld116328_RRRef
          WHERE T20._Fld67386=@P40
                AND T21._Fld67386=@P41
        )
            LEFT OUTER JOIN
            dbo._Reference65493 T22
            ON @P42=T22._IDRRef
               AND T22._Fld67386=@P43
        WHERE T18._Fld67386=@P44
              AND 0x08<>0x01
                  AND CASE
                        WHEN EXISTS
        (
          SELECT 0x01
          AS Q_002_F_000_
          FROM dbo._InfoRg100966 T23
          WHERE T23._Fld67386=@P45
                AND T23._Fld100967RRef=T18._IDRRef
                    AND T23._Fld100968_TYPE=0x08
                        AND T23._Fld100968_RTRef=0x0000FF7E
                        AND T23._Fld100968_RRRef=T1._Fld74270RRef
        )
                        THEN 0x01
                        ELSE 0x00
                      END=CASE
                            WHEN EXISTS
        (
          SELECT 0x01
          AS Q_003_F_000_
          FROM dbo._InfoRg100973 T24
          WHERE T24._Fld67386=@P46
                AND T24._Fld100974RRef=T18._IDRRef
                    AND T24._Fld100975_TYPE=0x08
                    AND T24._Fld100975_RTRef=0x0000FF7E
                    AND T24._Fld100976=0x00
        )
                            THEN 0x01
                            ELSE 0x00
                          END
                  AND 0x08<>0x01
                  AND NOT(NOT EXISTS
        (
          SELECT 0x01
          AS Q_004_F_000_
          FROM dbo._InfoRg99652 T25
          WHERE T25._Fld67386=@P47
                AND T25._Fld99653_TYPE=0x08
                    AND T25._Fld99653_RTRef=0x0000FFD5
                    AND T25._Fld99653_RRRef=T1._Fld74289RRef
                    AND T25._Fld99654_TYPE=0x08
                        AND T25._Fld99654_RTRef=0x0000FFD5
                        AND T25._Fld99654_RRRef=@P48
        )
                          AND NOT CASE
                                    WHEN EXISTS
        (
          SELECT top 356165165156165165 0x01
          AS Q_005_F_000_
          FROM dbo._InfoRg100966 T26
          INNER  JOIN                ---------------------------------------------------------------------------
          dbo._InfoRg99652 T27
          ON T26._Fld100967RRef=T18._IDRRef

             AND T26._Fld100968_TYPE=T27._Fld99654_TYPE
                 AND T26._Fld100968_RTRef=T27._Fld99654_RTRef
                 AND T26._Fld100968_RRRef=T27._Fld99654_RRRef

             AND T27._Fld99653_TYPE=0x08
                 AND T27._Fld99653_RTRef=0x0000FFD5
                 AND T27._Fld99653_RRRef=T1._Fld74289RRef
          WHERE T26._Fld67386=@P49
                AND T27._Fld67386=@P50
        )
                                    THEN 0x01
                                    ELSE 0x00
                                  END=CASE
                                        WHEN EXISTS
        (
          SELECT 0x01
          AS Q_006_F_000_
          FROM dbo._InfoRg100973 T28
          WHERE T28._Fld67386=@P51
                AND T28._Fld100974RRef=T18._IDRRef
                    AND T28._Fld100975_TYPE=0x08
                    AND T28._Fld100975_RTRef=0x0000FFD5
                    AND T28._Fld100976=0x00
        )
                                        THEN 0x01
                                        ELSE 0x00
                                      END)
                                  AND 0x08<>0x01
                                  AND CASE
                                        WHEN EXISTS
        (
          SELECT 0x01
          AS Q_007_F_000_
          FROM dbo._InfoRg100966 T29
          WHERE T29._Fld67386=@P52
                AND T29._Fld100967RRef=T18._IDRRef
                    AND T29._Fld100968_TYPE=0x08
                        AND T29._Fld100968_RTRef=0x000106FA
                        AND T29._Fld100968_RRRef=T1._Fld74272RRef
        )
                                        THEN 0x01
                                        ELSE 0x00
                                      END=CASE
                                            WHEN EXISTS
        (
          SELECT 0x01
          AS Q_008_F_000_
          FROM dbo._InfoRg100973 T30
          WHERE T30._Fld67386=@P53
                AND T30._Fld100974RRef=T18._IDRRef
                    AND T30._Fld100975_TYPE=0x08
                    AND T30._Fld100975_RTRef=0x000106FA
                    AND T30._Fld100976=0x00
        )
                                            THEN 0x01
                                            ELSE 0x00
                                          END
                                  AND CASE
                                        WHEN T1._Fld74272RRef=@P54
                                             OR T1._Fld74272RRef=@P55
                                        THEN CASE
                                               WHEN CASE
                                                      WHEN T14._Fld74350RRef IS NULL
                                                      THEN 0x01
                                                      ELSE CASE
                                                             WHEN T14._Fld74350RRef IS NOT NULL
                                                             THEN 0x08
                                                           END
                                                    END<>0x01
                                               THEN 0x01
                                               ELSE 0x00
                                             END
                                        WHEN T1._Fld74272RRef=@P56
                                        THEN CASE
                                               WHEN 0x08<>0x01
                                               THEN 0x01
                                               ELSE 0x00
                                             END
                                        ELSE 0x01
                                      END=0x01
                                  AND CASE
                                        WHEN T1._Fld74272RRef=@P57
                                             OR T1._Fld74272RRef=@P58
                                        THEN CASE
                                               WHEN 0x08<>0x01
                                                    AND CASE
                                                          WHEN EXISTS
        (
          SELECT 0x01
          AS Q_011_F_000_
          FROM dbo._InfoRg100966 T31
          WHERE T31._Fld67386=@P59
                AND T31._Fld100967RRef=T18._IDRRef
                    AND T31._Fld100968_TYPE=0x08
                        AND T31._Fld100968_RTRef=0x0001007C
                        AND T31._Fld100968_RRRef=ISNULL( T15._Fld116737RRef, @P60 )
        )
                                                          THEN 0x01
                                                          ELSE 0x00
                                                        END=CASE
                                                              WHEN EXISTS
        (
          SELECT 0x01
          AS Q_012_F_000_
          FROM dbo._InfoRg100973 T32
          WHERE T32._Fld67386=@P61
                AND T32._Fld100974RRef=T18._IDRRef
                    AND T32._Fld100975_TYPE=0x08
                    AND T32._Fld100975_RTRef=0x0001007C
                    AND T32._Fld100976=0x00
        )
                                                              THEN 0x01
                                                              ELSE 0x00
                                                            END
                                                    AND 0x08<>0x01
                                                        AND CASE
                                                              WHEN EXISTS
        (
          SELECT 0x01
          AS Q_013_F_000_
          FROM dbo._InfoRg100966 T33
          WHERE T33._Fld67386=@P62
                AND T33._Fld100967RRef=T18._IDRRef
                    AND T33._Fld100968_TYPE=0x08
                        AND T33._Fld100968_RTRef=0x00010103
                        AND T33._Fld100968_RRRef=ISNULL( T15._Fld116803RRef, @P63 )
        )
                                                              THEN 0x01
                                                              ELSE 0x00
                                                            END=CASE
                                                                  WHEN EXISTS
        (
          SELECT 0x01
          AS Q_014_F_000_
          FROM dbo._InfoRg100973 T34
          WHERE T34._Fld67386=@P64
                AND T34._Fld100974RRef=T18._IDRRef
                    AND T34._Fld100975_TYPE=0x08
                    AND T34._Fld100975_RTRef=0x00010103
                    AND T34._Fld100976=0x00
        )
                                                                  THEN 0x01
                                                                  ELSE 0x00
                                                                END
                                               THEN 0x01
                                               ELSE 0x00
                                             END
                                        ELSE CASE
                                               WHEN 0x08<>0x01
                                                    AND CASE
                                                          WHEN EXISTS
        (
          SELECT 0x01
          AS Q_009_F_000_
          FROM dbo._InfoRg100966 T35
          WHERE T35._Fld67386=@P65
                AND T35._Fld100967RRef=T18._IDRRef
                    AND T35._Fld100968_TYPE=0x08
                        AND T35._Fld100968_RTRef=0x0001007C
                        AND T35._Fld100968_RRRef=T1._Fld74288RRef
        )
                                                          THEN 0x01
                                                          ELSE 0x00
                                                        END=CASE
                                                              WHEN EXISTS
        (
          SELECT 0x01
          AS Q_010_F_000_
          FROM dbo._InfoRg100973 T36
          WHERE T36._Fld67386=@P66
                AND T36._Fld100974RRef=T18._IDRRef
                    AND T36._Fld100975_TYPE=0x08
                    AND T36._Fld100975_RTRef=0x0001007C
                    AND T36._Fld100976=0x00
        )
                                                              THEN 0x01
                                                              ELSE 0x00
                                                            END
                                               THEN 0x01
                                               ELSE 0x00
                                             END
                                      END=0x01
                                  AND (0x08<>0x01
                                       OR T1._Fld74284RRef=T22._Fld123105RRef)
      )
    )
    ORDER BY T1._Date_Time DESC, 
             T1._IDRRef DESC', 
         N'@P1 varbinary(16),@P2 varbinary(16),@P3 varbinary(16),@P4 varbinary(16),@P5 varbinary(16),@P6 varbinary(16),@P7 varbinary(16),@P8 varbinary(16),@P9 varbinary(16),@P10 numeric(10),@P11 numeric(10),@P12 numeric(10),@P13 numeric(10),@P14 numeric(10),@P15 numeric(10),@P16 numeric(10),@P17 varbinary(16),@P18 numeric(10),@P19 numeric(10),@P20 numeric(10),@P21 numeric(10),@P22 nvarchar(4000),@P23 numeric(10),@P24 varbinary(16),@P25 numeric(10),@P26 numeric(10),@P27 numeric(10),@P28 numeric(10),@P29 numeric(10),@P30 datetime2(3),@P31 numeric(10),@P32 numeric(10),@P33 numeric(10),@P34 numeric(10),@P35 numeric(10),@P36 numeric(10),@P37 nvarchar(4000),@P38 numeric(10),@P39 varbinary(16),@P40 numeric(10),@P41 numeric(10),@P42 varbinary(16),@P43 numeric(10),@P44 numeric(10),@P45 numeric(10),@P46 numeric(10),@P47 numeric(10),@P48 varbinary(16),@P49 numeric(10),@P50 numeric(10),@P51 numeric(10),@P52 numeric(10),@P53 numeric(10),@P54 varbinary(16),@P55 varbinary(16),@P56 varbinary(16),@P57 varbinary(16),@P58 varbinary(16),@P59 numeric(10),@P60 varbinary(16),@P61 numeric(10),@P62 numeric(10),@P63 varbinary(16),@P64 numeric(10),@P65 numeric(10),@P66 numeric(10)', 
         0xA586C641AA8D81AE4B56E687D0E70B4C, 
         0x87736B644D757E224405854295818758, 
         0xB24F430CD2009FAE4E22F484692215C8, 
         0x823E32DFA2BE22324A170D1F51AD2381, 
         0x82202EB29FA05EF541950D5CA07305C5, 
         0x87736B644D757E224405854295818758, 
         0xB24F430CD2009FAE4E22F484692215C8, 
         0x823E32DFA2BE22324A170D1F51AD2381, 
         0x82202EB29FA05EF541950D5CA07305C5, 
         0, 
         0, 
         0, 
         2, 
         1, 
         0, 
         0, 
         0x9EB3FD405C68C64C42B491437EDE1CAB, 
         1, 
         0, 
         1, 
         0, 
         N'РегистрНакопления.ДенежныеСредстваКВыплате', 
         0, 
         0xB8035480284F339E11E9440C701DA03C, 
         0, 
         0, 
         0, 
         0, 
         0, 
         '5999-11-01 00:00:00', 
         0, 
         0, 
         0, 
         0, 
         0, 
         0, 
         N'Документ.ЗаявкаНаРасходованиеДенежныхСредств', 
         0, 
         0xB8035480284F339E11E9440C701DA03C, 
         0, 
         0, 
         0xB8035480284F339E11E9440C701DA03C, 
         0, 
         0, 
         0, 
         0, 
         0, 
         0xB8035480284F339E11E9440C701DA03C, 
         0, 
         0, 
         0, 
         0, 
         0, 
         0xB96DB293C2F88D8C46DF79811923F574, 
         0xB4AF52C1B39555E54EEAC8D5724DC975, 
         0x87736B644D757E224405854295818758, 
         0xB96DB293C2F88D8C46DF79811923F574, 
         0xB4AF52C1B39555E54EEAC8D5724DC975, 
         0, 
         0x00000000000000000000000000000000, 
         0, 
         0, 
         0x00000000000000000000000000000000, 
         0, 
         0, 
         0;
  • Did you try statistics update with fullscan? – Yunus UYANIK Dec 18 '19 at 9:12
8

I would add OPTION(USE HINT 'DISABLE_OPTIMIZER_ROWGOAL') to the query to disable row goals.

The easiest way to spot the use of row goals is when using the TOP() operator, used to return 45 rows as quickly as possible in your case.

The use of loop joins is more common here as it is estimating smaller result sets & using non blocking operators.

Row goals will in most cases help, but with big convoluted queries I have seen them being bothersome in the past.

By disabling these row goals you should get a more optimized plan for the entire query.

You could also disable this by adding OPTION(QUERYTRACEON 4138) to the end of the query.


I tried to remove TOP 45 part - it works now less than 1 second. Where could I get more insight on row goals?

Removing the TOP(45) removed the row goal & improved performance in this case.

Paul White has some great blog posts on row goals. Parts 1 and 2 can be found here and here. There is also Inside the Optimizer: Row Goals In Depth.

If you cannot change the query, a plan guide could be added for the specific query to disable the row goal.


Another method to remove the row goal without adding the hint(s)

Following the statement:

A second general point to bear in mind is that a row goal is only set when the goal would be less than the regular estimate

Again from: Setting and Identifying Row Goals in Execution Plans

You could specify a nonsensical TOP() with such a high number that the row goal will not be set and then wrap your actual TOP() around it.

For example:

SELECT TOP(5) *
FROM(
SELECT TOP(99999999999) *
FROM dbo.table
ORDER BY column ASC
) AS  A;

More examples:

DB<>Fiddle

In short, the TOP(5) rowgoal is overriden by the much higher top(99999999999) in the inner query, resulting in only 2 actual row goals remaining in my example, one on the TOP(99999999) operator:

PhysicalOp="Top" LogicalOp="Top" EstimateRows="5" EstimateRowsWithoutRowGoal="9.4199"

And one on the Nested loops operator right after the second TOP() that exactly matches the EstimateRows

PhysicalOp="Nested Loops" LogicalOp="Inner Join" EstimateRows="9.4199" EstimateRowsWithoutRowGoal="9.4199"

enter image description here

For a real world example with a row goal issue such as yours, I was able to change the plan shape with non-blocking operators such as NL joins due to a TOP(5) row goal:

Actual performance issue not shown in screenshots

enter image description here

To hash match operators by using the TOP(999999999) method.
This plan shape matches the query when applying the row goal disable hints.

enter image description here

I would still prefer to add the query hint instead of writing the two TOP()s. But if you are not allowed to or can't, this could help.

| improve this answer | |

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.