How is LIKE implemented?

Question

Can anyone explain how the LIKE operator is implemented in current database systems (e.g. MySQL or Postgres)? or point me to some references that explain it?

The naive approach would be to inspect each record, executing a regular expression or partial string match on the field of interest, but I have a feeling (hope) that these systems do something smarter.

Answer 1

No, that's pretty much what they're doing. Now, if there is not a leading wildcard and the field is indexed, which is the usual situation, the database engine can apply the regular expression to the index. So, for example, if you write

SELECT *
  FROM employees
 WHERE last_name LIKE 'Cav%'

the database can use the index on LAST_NAME to find all the rows where the last name begins 'Cav'. On the other hand, if you had something like

SELECT *
  FROM employees
 WHERE last_name LIKE '%av%'

the database would have to scan the entire table (or the entire index) and evaluate the expression against the full LAST_NAME value. Obviously, that's very expensive.

Most of the better relational databases have facilities to do full-text search in a more efficient manner by constructing different sorts of indexes and text catalogs but these don't use the LIKE keyword. For example, here's a nice article that discusses full-text search in PostgreSQL.

Answer 2

In addition to what Justin Cave already wrote about PostgreSQL, there is a new feature in PostgreSQL 9.1 to speed up any search with LIKE (~~) or ILIKE (~~*). You can now use the operator classes provided by the module pg_trgm with a GIN or GiST index to speed up LIKE expressions that are not left-anchored. To install the extension, run once per database:

CREATE EXTENSION pg_trgm;

Create an index of the form

CREATE INDEX tbl_col_gin_trgm_idx ON tbl USING gin (col gin_trgm_ops);

Or:

CREATE INDEX tbl_col_gist_trgm_idx ON tbl USING gist (col gist_trgm_ops);

Creating and maintaining a GIN or GiST index carries a cost, but if your table is not heavily written, this is a great feature for you.

Depesz has written an excellent article in his blog about the new feature.

Answer 3

Speaking about MySQL, the position of the wild-card character (%) makes a difference. If the first part of the text is specified like where first_name like 'Sta%', then the DB engine will search only a smaller subset of words staring with S, then going to St, and then Sta, etc. If you do something like where first_name like '%stan%', then and entire scan of the column will be required. You can also look into full-text indexes that also does natural language searches. Check out the MySQL docs here.