Podcast #128: We chat with Kent C Dodds about why he loves React and discuss what life was like in the dark days before Git. Listen now.

Questions tagged [full-text-search]

Searching the text of a collection of documents or free text fields in a database to find those containing a word or combination of words.

Filter by
Sorted by
Tagged with
22
votes
3answers
8k views

How is LIKE implemented?

Can anyone explain how the LIKE operator is implemented in current database systems (e.g. MySQL or Postgres)? or point me to some references that explain it? The naive approach would be to inspect ...
5
votes
1answer
9k views

Mysql fulltext search my.cnf optimization

I have open a question on https://serverfault.com/questions/353888/mysql-full-text-search-cause-high-usage-cpu Some user recommended asking here. We built a news site. Every day we will input tens of ...
6
votes
2answers
2k views

MySQL EXPLAIN doesn't show 'use index' for FULLTEXT

I have a basic table : create table fullTextTest ( id INT(11) NOT NULL, superText CHAR(255) NOT NULL, superLongText TEXT NOT NULL, primary key (`id`), FULLTEXT KEY `superText` (`...
13
votes
1answer
9k views

Get partial match from GIN indexed TSVECTOR column

I would like to get results by query this: SELECT * FROM ( SELECT id, subject FROM mailboxes WHERE tsv @@ plainto_tsquery('avail') ) AS t1 ORDER by id DESC; This works and return rows with tsv ...
6
votes
1answer
2k views

FULLTEXT index ignored in BOOLEAN MODE with 'number of words' conditional

fulltext match is ignoring its index when I add a number of words conditional for its boolean mode. The selects are as follows: explain select * from seeds WHERE MATCH(text) AGAINST ("mount ...
8
votes
1answer
3k views

Can I have an InnoDB master and MyISAM slaves with Full-Text for searching?

I'd like to use InnoDB on the master, for its transactional ability, but MyISAM on the slaves for the full-text search ability. Is this possible?
4
votes
1answer
4k views

PostgreSQL FTS and Trigram-similarity Query Optimization

I have recently started working on PostgreSQL and I have around 12M rows to handle in which I want to apply Full Text Search. I don't have any prior experience in handling such databases. I have tried ...
8
votes
1answer
6k views

Best index for similarity function

So I have this table with 6.2 millions records and I have to perform search queries with similarity for one for the column. The queries can be: SELECT "lca_test".* FROM "lca_test" WHERE (...
7
votes
3answers
2k views

Finding the most commonly used non-stop words in a column

Simply put, I have a Postgres column containing ordinary prose and would like to determine the x most commonly used words ("word" being a group of characters delimited by a space, but not being stop ...
5
votes
3answers
2k views

Full Text Search With PostgreSQL

i have a table with this rows: Stickers ------------------------------------------------------ ID | Title |Keywords (ts_vector) ------------------------------------------------------ ...
3
votes
1answer
6k views

MySQL FullText search on string shorter than 3 chars returns no rows

I have a MySQL table with some addresses in it. Let's say I have the following 2 rows: "10 Fake Street" "101 Fake Street" I'm trying to use fulltext search with MATCH() AGAINST(). I have set ...
3
votes
1answer
3k views

Creating a case-insensitive and accent/diacritics insensitive search on a field

I would like to know if it makes sense to create an index combining two functions while using Full Text Search: lower(name) and f_unaccent(name) Where f_unaccent is just my wrapper to make unaccent ...
7
votes
1answer
2k views

How to search hyphenated words in PostgreSQL full text search?

I have to search for hyphenated words like 'good-morning', 'good-evening', etc. My query is: select id, ts_headline(content, to_tsquery('english','good-morning'), ...
5
votes
3answers
6k views

slow query with fulltext and left join

I have 3 tables about 250,000 records. table1 is MyISAM, add fullindex with title and content, add index with pid. table2 and table3 are InnoDB, add index with pid. Only query table3, just cost 0.04 ...
7
votes
2answers
3k views

Why does MyISAM support FULLTEXT Searching and InnoDB does not?

I know the basic differences between the MyISAM and InnoDB engines. In MySQL 5.6 beta, InnoDB will support fulltext search. But, I wonder why previous versions of InnobDB didn't support FTS? My guess ...
4
votes
1answer
1k views

FullTextSearch indexing/searching words with special characters

I'm trying to query FullTextSearch indexed column for hashtags SELECT Bio FROM Users WHERE CONTAINS (Bio,'promoter') SELECT Bio FROM Users WHERE CONTAINS (Bio,'#promoter') both results returns the ...
29
votes
1answer
7k views

Guidelines for full-text index maintenance

What guidelines should be considered for maintaining full-text indexes? Should I REBUILD or REORGANIZE the full-text catalog (see BOL)? What is a reasonable maintenance cadence? What heuristics (...
10
votes
1answer
35k views

Why is LIKE more than 4x faster than MATCH…AGAINST on a FULLTEXT index in MySQL?

I'm not getting this. I've got a table with these indexes PRIMARY post_id INDEX topic_id FULLTEXT post_text Table has (only) 346 000 rows. I am trying to perform 2 queries. SELECT ...
10
votes
1answer
12k views

Why full-text-search returns less rows than LIKE

I don't get full-text-search working as I want it to, and I don't understand the differences in the resultlists. Example statements: SELECT `meldungstext` FROM `artikel` WHERE `meldungstext` LIKE '%...
6
votes
2answers
6k views

PostgreSQL full text search on many columns

I need an advice with searching of record based on specified string. Search strings can contain values from these columns. Values in this string don´t have to be strictly identical given in the ...
10
votes
1answer
2k views

Calculate MATCH() AGAINST() scores from UNIFIED QUERY not FOR EACH TABLE

I am trying to have a score for entire section of SELECT statements SELECT *,MATCH(`result`) AGAINST('keyword') as `score` FROM `table1` WHERE MATCH(`result`) AGAINST('keyword') UNION SELECT *,MATCH(`...
10
votes
2answers
1k views

Slow fulltext search due to wildly inaccurate row estimates

Fulltext queries against this database (storing RT (Request Tracker) tickets) seem to be taking a very long time to execute. The attachments table (containing the fulltext data) is about 15GB. The ...
1
vote
1answer
2k views

Full text search on multiple joined tables

I'm trying to set up a full text search in PostgreSQL but coming across the problem that the more tables I join to the query the fewer the results I get back. The tables I am trying to join are: ...
4
votes
2answers
1k views

How to match all words of one column against words in another column

I have two tables. Table A: Company_name ------------------------------- Tata consultancy Infosys tech Atm service Air force firm Table B: Company1_name | Id -----------------...
40
votes
2answers
55k views

Possible INDEX on a VARCHAR field in MySql

I am working in a MySql database, with a table like this: +--------------+ | table_name | +--------------+ | myField | +--------------+ ...and I need to make a lot of queries like this (with ...
32
votes
5answers
17k views

What database technologies do big search engines use? [closed]

Does anybody knows how Google or Yahoo perform searches for keywords against very very huge amounts of data? What sort of database or technologies do they employ for this? It takes few milliseconds, ...
16
votes
2answers
1k views

Trigram search gets much slower as search string gets longer

In a Postgres 9.1 database, I have a table table1 with ~1.5M rows and a column label (simplified names for the sake of this question). There is a functional trigram-index on lower(unaccent(label)) (...
12
votes
4answers
10k views

Full text search results in a large amount of time spent in 'FULLTEXT initialization'

I'm currently trying to run some queries against a data dump of Stack Overflow's comments. Here's what the schema looks like: CREATE TABLE `socomments` ( `Id` int(11) NOT NULL, `PostId` int(11) ...
10
votes
2answers
4k views

Slow ORDER BY with LIMIT

I have this query: SELECT * FROM location WHERE to_tsvector('simple',unaccent2("city")) @@ to_tsquery('simple',unaccent2('wroclaw')) order by displaycount I'm happy with it: "Sort (cost=3842....
5
votes
2answers
2k views

MySQL fulltext selection performance

I have 1.5M rows in a table. Following is the table create code: CREATE TABLE `jobs` ( `id` INT(8) NOT NULL AUTO_INCREMENT, `job_id` VARCHAR(50) NOT NULL DEFAULT '', `title` VARCHAR(255) ...
13
votes
5answers
9k views

SQL Server 2008 Full Text Index Never Seems to Complete

Our website has a SQL Server 2008 R2 Express Edition database with full-text indexing for our website search. Each time a new record is added or updated in one of the indexed tables, the indexing ...
5
votes
2answers
13k views

How to set full text catalog location SQL Server 2008/2012

I want to upgrade my SQL Server 2005 database to SQL Server 2012. In SQL Server 2005 you can set the location of the full text catalog when you create it. But in SQL Server 2012 (and in SQL Server ...
5
votes
2answers
2k views

How can I implement full-text search on a single value in a JSONB column?

I have a table that stores a conversation between two people. The data will look something like this: CREATE TABLE foo AS SELECT $$[ { "user": 1, "timestamp": 1, "message": "First message" }, ...
4
votes
2answers
3k views

maintaining full text index on large table

I'm on SQL Server 2008 and have a table, for reporting purposes, with 500,000 records that will easily reach the millions. The table will employ a full text index for rapid searching on a handful of ...
3
votes
3answers
1k views

PostgreSQL GIN index not used when ts_query language is fetched from a column

I've got a table that stores some multilingual content: CREATE TABLE search ( content text NOT NULL, language regconfig NOT NULL, fulltext tsvector ); CREATE INDEX search_fulltext ON search ...
3
votes
1answer
3k views

How to make a FULLTEXT search with ORDER BY fast?

I'm trying to get a simple FULLTEXT match to be faster when using order by on another column on a table with over 100 million rows. The basis is one table with a fulltext on two columns and I want to ...
2
votes
1answer
638 views

Full text search on two tsvector columns

I'm trying to do a full text search across two columns of two separate tables in PostgreSQL 9.4.2. The tables and columns are: article.article_title keyword.keyword Where there is a many-to-many ...
2
votes
2answers
4k views

MySQL - SQL statement to test minimum length of the word 'ft_min_word_len'

I'm trying to understand FULLTEXT indexes. Is there some SQL query to test 'ft_min_word_len' variable? I've tried to match four characters only in the example below but it doesn't work. Why the ...
2
votes
1answer
375 views

“Wrong affix file formate for flag” while loading dictionaries from LibreOffice in PostgreSQL 9.3

Trying to get good TEXT SEARCH results out of PostgreSQL in English, German and Dutch made that I downloaded proofing tools of LibreOffice (tried of version 4.3 and 3.3) and iSpell. I placed the files ...
2
votes
2answers
1k views

MySQL Full Text index issue

I Have a table articles with following data mysql> select * from articles; +----+-----------------------+------------------------------------------+ | id | title | body ...
1
vote
1answer
6k views

mysql MATCH relavancy score

I thought this would be an easy question to google for; but I couldn't not find it. Given the following query: SELECT name, MATCH (name) AGAINST ('vision black*' IN BOOLEAN MODE) as rel FROM (`...
0
votes
1answer
2k views

Mixed search in multiple columns

I have a table which has a particular column of type JSON, where some user's contact information are stored. I can alter the table and add new columns with cached values to increase search performance ...
4
votes
1answer
576 views

Mysql fulltext boolean ignore phrase

I'm trying to come up with a mysql boolean search that matches "monkey" but ignores / doesn't take into account any matches of "monkey business". Something like the following is the general idea but ...
4
votes
2answers
2k views

Slow insert with MySQL full-text index

I use a full-text index in a MySQL table, and each insert into this table takes about 3 seconds. It seems that MySQL rebuilds (a part) of the full text index after each insert/update. Is this right? ...
3
votes
1answer
750 views

Can underscore be forced as a word splitter without a full-text parser plugin?

Is there any way in MySQL to force underscore to become a word splitter character (like space or dash) without having to use a fulltext parser plugin? If not, does such a plugin already exist or do I ...
3
votes
2answers
4k views

Oracle “matches” multiple columns

I am developing a web service where a term of search is given, then I return all the matches from employees table. In MySQL I used to do the following: SELECT * FROM employees WHERE MATCH(name, ...
2
votes
1answer
4k views

MySQL FULLTEXT search not returning any results

I'm wondering why my fulltext search isn't returning any results. To me this seems pretty straightforward, but alas it's not working. I'm far from a DBA, but looking at the EXPLAINed query, I see ...
2
votes
1answer
2k views

Reorganize full text catalog is offline or online?

I need to schedule a Full Text Search Maintenance and I was reading this link: http://msdn.microsoft.com/en-us/library/ms176095.aspx. Reading this documentation I am supposing that I need to rebuild/...
2
votes
0answers
358 views

ts_headline() not highlighting properly for <-> operator

ts_headline() does not seem to highlight the input text in a way that respects the <-> operator. The following query: SELECT ts_headline('English','This Commercial Bank does not have any ...
2
votes
1answer
119 views

SQL Server FullText search on non-persisted columns

Let's say I have a table with 3 columns: Id, ColA and ColB. Then I created a FullText index: CREATE FULLTEXT INDEX ON [Table] ( ColA LANGUAGE 1033, ColB LANGUAGE 1033 ) KEY INDEX [...