1

Background: I've been using Postgres full-text search and it has met my needs quite well. Though there is some unexpected behavior that I cannot seem to wrap my head around. It has to do with the full-text search results returning the highlighted matches using the ts_headline function. It returns the correct matches most of the time, but often it will not return a match as I expect it. I think an example is the best way to demonstrate this.

Relevant Postgres full-text highlighting docs: https://www.postgresql.org/docs/current/textsearch-controls.html#TEXTSEARCH-HEADLINE

Versions: I have tried Postgres 15 and 12 and experienced this bug (feature?) in both.

Examples: In the following 3 full-text search highlighting queries, why do the first one's results not match the second and third's? Im trying to figure out what I can do to get matches on the first query. I first noticed that some of my highlight queries were coming back with no 'hits'. I created the following 3 queries to show the issue I'm having.

According to the docs, when there are no matches (identified by <b></b> tags) it simply returns the first MinWords. That's what is happening in the first query when I think that we should actually get the 2 results back as we do in the following two queries.

postgres=# SELECT ts_headline('english', 'beginning word word word word CHILD word word word word word word word word word word word word word word word word word word word word word word word word word word word word word word word word word word word word word word word word word word word word word word word word word word word word word word word word word word word word word word word word word word word word word word word word word word word word word word word word word word word word word word word word word word word word word word word word word word word SERVICE ending', 
    to_tsquery('english', 'CHILD & SERVICE'), 
    'MaxFragments=2,MinWords=5,MaxWords=10');

          ts_headline          
-------------------------------
 beginning word word word word
(1 row)

Now let's increase MaxWords, now we get 2 matches


postgres=# SELECT ts_headline('english', 'beginning word word word word CHILD word word word word word word word word word word word word word word word word word word word word word word word word word word word word word word word word word word word word word word word word word word word word word word word word word word word word word word word word word word word word word word word word word word word word word word word word word word word word word word word word word word word word word word word word word word word word word word word word word word word SERVICE ending', 
    to_tsquery('english', 'CHILD & SERVICE'), 
    'MaxFragments=2,MinWords=5,MaxWords=11');

                                                      ts_headline                                                       
------------------------------------------------------------------------------------------------------------------------
 beginning word word word word <b>CHILD</b> word word word word word ... word word word word word <b>SERVICE</b> ending
(1 row)

Now let's increase MaxFragments, now we get 2 matches

postgres=# SELECT ts_headline('english', 'beginning word word word word CHILD word word word word word word word word word word word word word word word word word word word word word word word word word word word word word word word word word word word word word word word word word word word word word word word word word word word word word word word word word word word word word word word word word word word word word word word word word word word word word word word word word word word word word word word word word word word word word word word word word word word SERVICE ending', 
    to_tsquery('english', 'CHILD & SERVICE'), 
    'MaxFragments=3,MinWords=5,MaxWords=10');

                                               ts_headline                                               
---------------------------------------------------------------------------------------------------------
 word word word word <b>CHILD</b> word word word word word ... word word word word <b>SERVICE</b> ending
(1 row)

I feel like something subtle is going on between all the MaxFragments, MinWords, MaxWords settings, or maybe this is undefined behavior or a bug. Im hoping to find a way to get the first query to match as I do believe it should. Please correct me if I'm wrong.

1 Answer 1

0

Awkwardly, this appears to have to do with the fact that you have repeated the word term repeatedly, which appears to be choking the lookup logic.

Witness that if I insert word2 for every second word, you receive more interesting results:

SELECT ts_headline('english', 'beginning word word word word CHILD word word2 word word2 word word2 word word2 word word2 word word2 word word2 word word2 word word2 word word2 word word2 word word2 word word2 word word2 word word2 word word2 word word2 word word2 word word2 word word2 word word2 word word2 word word2 word word2 word word2 word word2 word word2 word word2 word word2 word word2 word word2 word word2 word word2 word word2 word word2 word word2 word word2 word word2 word word2 word word2 word word2 word word2 word word2 word word2 word word2 word word2 SERVICE ending', 
    to_tsquery('english', 'CHILD & SERVICE'), 
    'MaxFragments=2,MinWords=5,MaxWords=10');

ts_headline
word word word word <b>CHILD</b> word word2 word word2 word ... word word2 word word2 <b>SERVICE</b> ending

That said, I would argue that this is in fact a bug and should be raised, because of the remedy of setting MaxFragments=3. This feels off-by-one :)

Also, if one replaces word with sword (5 letters) in your example, we also get better results:

SELECT ts_headline('english', 'beginning word word word word CHILD sword sword sword sword sword sword sword sword sword sword sword sword sword sword sword sword sword sword sword sword sword sword sword sword sword sword sword sword sword sword sword sword sword sword sword sword sword sword sword sword sword sword sword sword sword sword sword sword sword sword sword sword sword sword sword sword sword sword sword sword sword sword sword sword sword sword sword sword sword sword sword sword sword sword sword sword sword sword sword sword sword sword sword sword SERVICE ending', 
    to_tsquery('english', 'CHILD & SERVICE'), 
    'MaxFragments=2,MinWords=5,MaxWords=10');
ts_headline
word word word word <b>CHILD</b> sword sword sword sword sword ... sword sword sword sword <b>SERVICE</b> ending

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.