I have a table with simple structure (id, metadata_json, stamp), stamp is a timestamp and has Btree index on it. MetadataJson is a jsonb with GIN index.
The table has 25M rows. I am using PostgreSQL 10.
The query I am executing is quite simple:
select * from metadata where metadata_json @> '{"someBool": true}'
AND stamp >= '01-01-2016' ORDER BY stamp DESC LIMIT 100;
My idea how it should work: I have a btree on stamp, hence it should go by the index in reverse order, then it should test rows on the json restriction (it has selectivity 40%). I would expect it to return in few milliseconds.
QUERY PLAN
----------------------------------------------------------------------------------------------------------------------------------------------------------
Limit (cost=80598.46..80598.71 rows=100 width=381) (actual time=445064.728..445064.791 rows=100 loops=1)
-> Sort (cost=80598.46..80607.46 rows=3600 width=381) (actual time=445064.724..445064.754 rows=100 loops=1)
Sort Key: stamp DESC
Sort Method: top-N heapsort Memory: 109kB
-> Bitmap Heap Scan on metadata (cost=66591.00..80460.87 rows=3600 width=381) (actual time=2881.164..444283.520 rows=1437024 loops=1)
Recheck Cond: ((metadata_json @> '{"someBool": true}'::jsonb) AND (stamp >= '2016-01-01 00:00:00'::timestamp without time zone))
Heap Blocks: exact=882439
-> BitmapAnd (cost=66591.00..66591.00 rows=3600 width=0) (actual time=2599.415..2599.415 rows=0 loops=1)
-> Bitmap Index Scan on metadata_idx (cost=0.00..260.25 rows=25100 width=0) (actual time=1762.166..1762.166 rows=10041746 loops=1)
Index Cond: (metadata_json @> '{"someBool": true}'::jsonb)
-> Bitmap Index Scan on stamp_idx (cost=0.00..66328.69 rows=3600034 width=0) (actual time=760.136..760.136 rows=3591329 loops=1)
Index Cond: (stamp >= '2016-01-01 00:00:00'::timestamp without time zone)
Planning time: 5.008 ms
Execution time: 445072.043 ms
(14 rows)
From the plan it seems that the planner has statistics really off, but the table is analyzed, sampling was set to 1000.
Edit: after some searching I found out that postgres does not have statistics for jsonb data type... Can you please hint me how this type of query can be optimized?