I have a very large table (around 140M rows) with five columns:
CREATE TABLE IF NOT EXISTS citations
(
citing_id text,
citing_issn text,
cited_id text,
cited_issn text,
year int
);
There are hash indexes on citing_id
and on cited_id
, neither of which has any NULL values (confirmed), but queries selecting on equality in these columns are returning zero results no matter what. Even this query, which I would think would work in any populated table, returns zero rows:
SELECT *
FROM citations
WHERE citing_id IN (SELECT citing_id
FROM citations LIMIT 1)
LIMIT 100;
However, the same query on a non-indexed column works exactly as expected:
SELECT *
FROM citations
WHERE citing_issn IN (SELECT citing_issn
FROM citations LIMIT 1)
LIMIT 100;
The data was loaded from a CSV file using the COPY
command, and I suspect there may be extra hidden characters on the end of the indexed values -- querying with WHERE citing_id LIKE '<id>%'
is very slow (hash indexes don't help with partial strings) but returns the right results. Still, the first query above should work no matter what, unless I'm mistaken.
I'm running Postgres 9.5.3.0 on MacOS 10.13.1
edit:
The query plan for the first query (on the indexed column, returning no rows) is:
Limit (cost=0.03..1.38 rows=100 width=58)
-> Nested Loop (cost=0.03..265.83 rows=19751 width=58)
-> HashAggregate (cost=0.03..0.04 rows=1 width=17)
Group Key: citations_1.citing_id
-> Limit (cost=0.00..0.02 rows=1 width=17)
-> Seq Scan on citations citations_1 (cost=0.00..3164425.68 rows=148131768 width=17)
-> Index Scan using citations_citing_id on citations (cost=0.00..265.14 rows=65 width=58)
Index Cond: (citing_id = citations_1.citing_id)
and for the second query (on the non-indexed column, returning expected results):
Limit (cost=0.04..139.54 rows=100 width=58)
-> Hash Semi Join (cost=0.04..3581836.35 rows=2567617 width=58)
Hash Cond: (citations.citing_issn = citations_1.citing_issn)
-> Seq Scan on citations (cost=0.00..3164425.68 rows=148131768 width=58)
-> Hash (cost=0.03..0.03 rows=1 width=10)
-> Limit (cost=0.00..0.02 rows=1 width=10)
-> Seq Scan on citations citations_1 (cost=0.00..3164425.68 rows=148131768 width=10)
SELECT citing_id FROM citations LIMIT 1
cannot return NULL ?SELECT citing_id FROM citations LIMIT 1
returns exactly one row with a non-NULL value. There are no NULL values in the table.SELECT COUNT(*), COUNT(citing_id) FROM citations;
Even one row withNULL
inciting_id
can make your first query return 0 rows.SELECT * FROM citations WHERE citing_id IS NOT DISTINCT FROM (SELECT citing_id FROM citations LIMIT 1) ;