Slow insert with MySQL full-text index

Question

I use a full-text index in a MySQL table, and each insert into this table takes about 3 seconds. It seems that MySQL rebuilds (a part) of the full text index after each insert/update. Is this right?

The table has about 30 varchar and text fields, some indexes on varchar-fields and a fulltext index over the three fields title_text (char(255)), content_text (text), author_text (char(255)). In the table are about 210.000 rows, the table size is about 2.5 GB and the index size about 0.5 GB. The insert queries are done for every new row (no combined query for all inserts).

How can I get better performance from the INSERT? Is there perhaps an option to set when MySQL rebuilds the full-text index?

Answer 1

It must be as quick if not quicker to do:

1. ALTER TABLE mydb.datafulltxt DISABLE KEYS;
2. Do many INSERTs into mydb.datafulltxt
3. ALTER TABLE mydb.datafulltxt ENABLE KEYS;
4. Repeat as required.

Answer 2

From the MySQL 5.7 Reference Manual, 8.2.4.1 Optimizing INSERT Statements:

To optimize insert speed, combine many small operations into a single large operation. Ideally, you make a single connection, send the data for many new rows at once, and delay all index updates and consistency checking until the very end.

The time required for inserting a row is determined by the following factors, where the numbers indicate approximate proportions:

• Connecting: (3)

• Sending query to server: (2)

• Parsing query: (2)

• Inserting row: (1 × size of row)

• Inserting indexes: (1 × number of indexes)

• Closing: (1)

This does not take into consideration the initial overhead to open tables, which is done once for each concurrently running query.

The size of the table slows down the insertion of indexes by log N, assuming B-tree indexes.

You can use the following methods to speed up inserts:

• If you are inserting many rows from the same client at the same time, use INSERT statements with multiple VALUES lists to insert several rows at a time. This is considerably faster (many times faster in some cases) than using separate single-row INSERT statements. If you are adding data to a nonempty table, you can tune the bulk_insert_buffer_size variable to make data insertion even faster.

• When loading a table from a text file, use LOAD DATA INFILE. This is usually 20 times faster than using INSERT statements.

• Take advantage of the fact that columns have default values. Insert values explicitly only when the value to be inserted differs from the default. This reduces the parsing that MySQL must do and improves the insert speed.