Skip to main content
deleted 4 characters in body
Source Link

Even SQL Server doesn't use min_data_id or max_data_id directly when processing MIN or MAX aggregates, even when aggregate pushdown (SQL Server 2016 or later, Enterprise Edition or equivalent) is employed. Every row in qualifying buckets is processed, albeit using an efficient SIMD implementation. To illustrate, the following stack trace was obtained on SQL Server 2022:

Pushed down aggregate stack trace

This doesn't necessarily mean the metadata is unreliable, though it could be if the deleted bitmap is non-empty since the min or max row might be logically deleted. There is also the question of which rows are visible to the current transaction given the current isolation level. Whatever the exact reasons, the fact remains that SQL Server itself does not employ this potential optimisation.

You could decide that approximate results are good enough for your needs. Perhaps you know delta stores and the deleted bitmap will always be empty. You'll still need to decode the bigint values of min_data_id and max_data_id to the target data type when value-based encoding is used.

When hash-based encoding is used with a dictionary, you're simply out of luck. There's no way supported way to access the dictionary from T-SQL, much less a way to decode a particular value to a correctly typed dictionary entry.

In your position, I would be more concerned with getting the best possible supported performance for your MIN and MAX aggregates. Aggregate pushdown is a must, but that does depend on the data types in use and your version and edition of SQL Server, among other factors.

You would start by looking for the Actual Number of Locally Aggregated Rows on the columnstore scan in a post-execution ('actual') execution plan. Beyond that, you would look into the amount of physical I/O involved and sizing of the columnstore memory pool, along with CPU utilisation. Under ideal conditions, SQL Server is capable of extremely good aggregate performance on columnstore data.

Even SQL Server doesn't use min_data_id or max_data_id directly when processing MIN or MAX aggregates, even when aggregate pushdown (SQL Server 2016 or later, Enterprise Edition or equivalent) is employed. Every row in qualifying buckets is processed, albeit using an efficient SIMD implementation. To illustrate, the following stack trace was obtained on SQL Server 2022:

Pushed down aggregate stack trace

This doesn't necessarily mean the metadata is unreliable, though it could be if the deleted bitmap is non-empty since the min or max row might be logically deleted. There is also the question of which rows are visible to the current transaction given the current isolation level. Whatever the exact reasons, the fact remains that SQL Server itself does not employ this potential optimisation.

You could decide that approximate results are good enough for your needs. Perhaps you know delta stores and the deleted bitmap will always be empty. You'll still need to decode the bigint values of min_data_id and max_data_id to the target data type when value-based encoding is used.

When hash-based encoding is used with a dictionary, you're simply out of luck. There's no way supported way to access the dictionary from T-SQL, much less a way to decode a particular value to a correctly typed dictionary entry.

In your position, I would be more concerned with getting the best possible supported performance for your MIN and MAX aggregates. Aggregate pushdown is a must, but that does depend on the data types in use and your version and edition of SQL Server, among other factors.

You would start by looking for the Actual Number of Locally Aggregated Rows on the columnstore scan in a post-execution ('actual') execution plan. Beyond that, you would look into the amount of physical I/O involved and sizing of the columnstore memory pool, along with CPU utilisation. Under ideal conditions, SQL Server is capable of extremely good aggregate performance on columnstore data.

Even SQL Server doesn't use min_data_id or max_data_id directly when processing MIN or MAX aggregates, even when aggregate pushdown (SQL Server 2016 or later, Enterprise Edition or equivalent) is employed. Every row in qualifying buckets is processed, albeit using an efficient SIMD implementation. To illustrate, the following stack trace was obtained on SQL Server 2022:

Pushed down aggregate stack trace

This doesn't necessarily mean the metadata is unreliable, though it could be if the deleted bitmap is non-empty since the min or max row might be logically deleted. There is also the question of which rows are visible to the current transaction given the current isolation level. Whatever the exact reasons, the fact remains that SQL Server itself does not employ this potential optimisation.

You could decide that approximate results are good enough for your needs. Perhaps you know delta stores and the deleted bitmap will always be empty. You'll still need to decode the bigint values of min_data_id and max_data_id to the target data type when value-based encoding is used.

When hash-based encoding is used with a dictionary, you're simply out of luck. There's no supported way to access the dictionary from T-SQL, much less a way to decode a particular value to a correctly typed dictionary entry.

In your position, I would be more concerned with getting the best possible supported performance for your MIN and MAX aggregates. Aggregate pushdown is a must, but that does depend on the data types in use and your version and edition of SQL Server, among other factors.

You would start by looking for the Actual Number of Locally Aggregated Rows on the columnstore scan in a post-execution ('actual') execution plan. Beyond that, you would look into the amount of physical I/O involved and sizing of the columnstore memory pool, along with CPU utilisation. Under ideal conditions, SQL Server is capable of extremely good aggregate performance on columnstore data.

Source Link
Paul White
  • 90.3k
  • 30
  • 424
  • 663

Even SQL Server doesn't use min_data_id or max_data_id directly when processing MIN or MAX aggregates, even when aggregate pushdown (SQL Server 2016 or later, Enterprise Edition or equivalent) is employed. Every row in qualifying buckets is processed, albeit using an efficient SIMD implementation. To illustrate, the following stack trace was obtained on SQL Server 2022:

Pushed down aggregate stack trace

This doesn't necessarily mean the metadata is unreliable, though it could be if the deleted bitmap is non-empty since the min or max row might be logically deleted. There is also the question of which rows are visible to the current transaction given the current isolation level. Whatever the exact reasons, the fact remains that SQL Server itself does not employ this potential optimisation.

You could decide that approximate results are good enough for your needs. Perhaps you know delta stores and the deleted bitmap will always be empty. You'll still need to decode the bigint values of min_data_id and max_data_id to the target data type when value-based encoding is used.

When hash-based encoding is used with a dictionary, you're simply out of luck. There's no way supported way to access the dictionary from T-SQL, much less a way to decode a particular value to a correctly typed dictionary entry.

In your position, I would be more concerned with getting the best possible supported performance for your MIN and MAX aggregates. Aggregate pushdown is a must, but that does depend on the data types in use and your version and edition of SQL Server, among other factors.

You would start by looking for the Actual Number of Locally Aggregated Rows on the columnstore scan in a post-execution ('actual') execution plan. Beyond that, you would look into the amount of physical I/O involved and sizing of the columnstore memory pool, along with CPU utilisation. Under ideal conditions, SQL Server is capable of extremely good aggregate performance on columnstore data.