I want to perform nearest neighbors search on db with 5000 entries of cube array with 512 dimensions.

I referred to this answer and used docker image of postgresql with cube array limit set to 2048 which can be found in the comment to the answer. But I'm getting the following error:

ERROR: failed to add item to index page in "faq_c_idx" SQL state: XX000

It works with dimensions=~200, but when I set higher dimensions I keep receiving the error above.

Here's the query:

  SELECT i, cube(array_agg(random()::float)) AS c
  FROM generate_series(1,5000) AS i
  CROSS JOIN LATERAL generate_series(1,512)
    AS x

CREATE INDEX ON mytable USING gist ( c );
ANALYZE mytable;

It can't find a place to store the index tuple, probably because the tuple is larger than a page and so can't fit on any page. You could recompile with a larger blocksize, e.g. ./configure --with-blocksize=32, but then you will just run into a different limitation, probably INDEX_SIZE_MASK, which is not so easy to work around.

So, you just can't do this. But if you could, an index into a 512 dimensional cube is not likely to be all that useful anyway. It will probably be slower than a seq-scan would be, or at least not faster by very much.

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.