I just setup a new AWS RDS postgres server and queries are painfully slow (> 10 hours) such that most of the time I just end up killing the query. I have done some indexing that cut the EXPLAIN cost down by a factor of ten but it is still just as slow. This is a migration from redshift where the same query used to take about 5minutes to complete so I would've expected this to be around the same time since redshift only has one sort key.
Running AWS RDS Postgres 9.5.2 on a db.r3.8xlarge (32vCPU, 244gb RAM), work_mem=1GB
.
Query
SELECT *
FROM
(SELECT *
FROM table1
WHERE create_date >= '2015-10-01' AND create_date <= '2015-12-31')
JOIN
(SELECT *
FROM table2
WHERE type = 6)
ON table1.id = table2.id
WHERE table1.indicator1 = 'N' AND table1.product = ‘foo’ AND table1.indicator2 = ‘Y’
EXPLAIN Output
Nested Loop (cost=109063.49..593806.97 rows=20886 width=91)
-> Bitmap Heap Scan on table1 (cost=109062.91..255339.43 rows=39386 width=80)
Recheck Cond: (((product)::text = ‘foo’::text) AND (indicator2 = ‘Y’::bpchar) AND (indicator1 = 'N'::bpchar) AND (create_date >= '2015-10-01'::date) AND (create_date <= '2015-12-31'::date))
-> BitmapAnd (cost=109062.91..109062.91 rows=39386 width=0)
-> Bitmap Index Scan on multi_no_date_idx (cost=0.00..22893.13 rows=662445 width=0)
Index Cond: (((product)::text = ‘foo’::text) AND (indicator2 = ‘Y’::bpchar) AND (indicator1 = 'N'::bpchar))
-> Bitmap Index Scan on create_date_idx (cost=0.00..86149.84 rows=4108527 width=0)
Index Cond: ((create_date >= '2015-10-01'::date) AND (create_date <= '2015-12-31'::date))
-> Index Scan using table2_pkey on table2 (cost=0.57..8.57 rows=1 width=19)
Index Cond: ((id = table1.id) AND (type = 1))
(10 rows)
Table1 Indices
Indexes:
“table1_pkey" PRIMARY KEY, btree (id)
“indicator1_idx" btree (indicator1)
“create_date_idx" btree (create_date)
“indicator2” btree (indicator2)
“table1_id_idx" btree (id)
“product_idx” btree (product)
"multi_no_date_idx" btree (indicator1, indicator2, product)
Table2 Indices
Indexes:
“table2_pkey" PRIMARY KEY, btree (id, type)
“id_idx" btree (id)
XN Hash Join DS_DIST_NONE (cost=7840.77..20461918.52 rows=1 width=312) Hash Cond: ("outer".id = "inner".id) -> XN Seq Scan on table2 (cost=0.00..20276990.40 rows=14166984 width=27) Filter: (type = 1) -> XN Hash (cost=7840.77..7840.77 rows=1 width=298) -> XN Seq Scan on table1 (cost=0.00..7840.77 rows=1 width=298) Filter: ((create_date <= '2015-12-31'::date) AND (create_date >= '2015-10-01'::date) AND ((product)::text = ‘foo’::text) AND (indicator2 = ‘Y’::bpchar) AND (indicator1 = 'N'::bpchar))