1

I use Pg FTS to text-search database containing some technical descriptions/comments. Which in particular happen to contain XML snippets. Those are amazingly difficult to find by XML tags. I found basic workaround (to find tags exactly-as-written) but look for suggestions how to improve.

Default state

CREATE TABLE mytab(id INTEGER, descr TEXT);
INSERT INTO mytab VALUES(1, 'Please create table');
INSERT INTO mytab VALUES(2, 'Please add <some-tag/> to config file');

SELECT id FROM mytab where descr @@ 'add';
    2
   (1 row)

SELECT id FROM mytab where descr @@ '<some-tag/>';
   (0 rows)

SELECT id FROM mytab where descr @@ 'some-tag';
   (0 rows)

SELECT ts_debug('add <some-tag/>');
   (asciiword,"Word, all ASCII",add,{english_stem},english_stem,{add})
   (blank,"Space symbols"," ",{},,)
   (tag,"XML tag",<some-tag/>,{},,)

Finding tags as exactly-written

I managed to improve things a little bit by:

CREATE TEXT SEARCH CONFIGURATION xmlfixed(COPY = english);
ALTER TEXT SEARCH CONFIGURATION xmlfixed ADD MAPPING FOR TAG WITH simple;

SELECT id FROM mytab WHERE TO_TSVECTOR('xmlfixed', descr) @@ TO_TSQUERY('xmlfixed', 'add');
      2
    (1 row)

SELECT id FROM mytab WHERE TO_TSVECTOR('xmlfixed', descr) @@ TO_TSQUERY('xmlfixed', '<some-tag/>');
      2
    (1 row)

but this works only if tag is written exactly. I still have

SELECT id FROM mytab WHERE TO_TSVECTOR('xmlfixed', descr) @@ TO_TSQUERY('xmlfixed', '<some-tag>');
    (0 rows)
SELECT id FROM mytab WHERE TO_TSVECTOR('xmlfixed', descr) @@ TO_TSQUERY('xmlfixed', 'some-tag');
    (0 rows)

(ok, ok, internally there were GIN indexes and reindex, but that's orthogonal)

Looking for help

Is there any way to configure this search so record 2 can be found when querying user asks for some-tag? Is it possible without writing new binary extension?

I suppose I could try implementing new custom dictionary which would strip <>/ characters, and use it together with simple as mapping for tag (codewise sth similar to https://stackoverflow.com/questions/27993032/custom-postgres-fts-dictionary, and above that ALTER TEXT SEARCH CONFIGURATION xmlfixed ALTER MAPPING FOR TAG WITH xmlstripper, simple;, where xmlstripper would be this new extension). Still, distributing new binary extension seems very difficult (if at all possible) at organizational level.

Mayhaps there is some other way?

New contributor
Mekk is a new contributor to this site. Take care in asking for clarification, commenting, and answering. Check out our Code of Conduct.

Your Answer

Mekk is a new contributor. Be nice, and check out our Code of Conduct.

By clicking "Post Your Answer", you acknowledge that you have read our updated terms of service, privacy policy and cookie policy, and that your continued use of the website is subject to these policies.

Browse other questions tagged or ask your own question.