I have a table with 7.2 million tuples which looks like this:
table public.methods
column | type | attributes
--------+-----------------------+----------------------------------------------------
id | integer | not null DEFAULT nextval('methodkey'::regclass)
hash | character varying(32) | not null
string | character varying | not null
method | character varying | not null
file | character varying | not null
type | character varying | not null
Indexes:
"methods_pkey" PRIMARY KEY, btree (id)
"methodhash" btree (hash)
Now I want to select some values but the query is incredibly slow:
db=# explain
select hash, string, count(method)
from methods
where hash not in
(select hash from nostring)
group by hash, string
order by count(method) desc;
QUERY PLAN
----------------------------------------------------------------------------------------
Sort (cost=160245190041.10..160245190962.07 rows=368391 width=182)
Sort Key: (count(methods.method))
-> GroupAggregate (cost=160245017241.77..160245057764.73 rows=368391 width=182)
-> Sort (cost=160245017241.77..160245026451.53 rows=3683905 width=182)
Sort Key: methods.hash, methods.string
-> Seq Scan on methods (cost=0.00..160243305942.27 rows=3683905 width=182)
Filter: (NOT (SubPlan 1))
SubPlan 1
-> Materialize (cost=0.00..41071.54 rows=970636 width=33)
-> Seq Scan on nostring (cost=0.00..28634.36 rows=970636 width=33)
The hash
column is the md5 hash of string
and has an index. So I think my problem is that the whole table is sorted by id and not by hash, so it takes a while to sort it first and then group it?
The table nostring
contains only a list of hashes I don't want to have. But I need both tables to have all values. So it's not an option to delete these.
additional info: none of the columns can be null (fixed that in the table definition) and i'm using postgresql 9.2.