0

Hive Materialized Views support partitions,

CREATE MATERIALIZED VIEW [IF NOT EXISTS] [db_name.]materialized_view_name
  [DISABLE REWRITE]
  [COMMENT materialized_view_comment]
  [PARTITIONED ON (col_name, ...)]
  [
    [ROW FORMAT row_format]
    [STORED AS file_format]
      | STORED BY 'storage.handler.class.name' [WITH SERDEPROPERTIES (...)]
  ]
  [LOCATION hdfs_path]
  [TBLPROPERTIES (property_name=property_value, ...)]
AS
<query>;

Hive supports "incremental changes",

Hive supports incremental view maintenance, i.e., only refresh data that was affected by the changes in the original source tables. Incremental view maintenance will decrease the rebuild step execution time. In addition, it will preserve LLAP cache for existing data in the materialized view.

From the Release Notes,

[HIVE-18839] - Implement incremental rebuild for materialized views (only insert operations in source tables)

[HIVE-18321] - Support REBUILD for MVs backed by custom storage handlers

But it's not clear if that will work with partitions.

If my source tables used in <query> have a partition added to them, will the MATERIALIZED VIEW update for the new partitions efficiently?

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Browse other questions tagged or ask your own question.