Performance implications of MySQL VARCHAR sizes

Question

Is there a performance difference in MySQL between varchar sizes? For example, varchar(25) and varchar(64000). If not, is there a reason not to declare all varchars with the max size just to ensure you don't run out of room?

Answer 1

You must realize the tradeoffs of using CHAR vs VARCHAR

With CHAR fields, what you allocate is exactly what you get. For example, CHAR(15) allocates and stores 15 bytes, no matter how characters you place in the field. String manipulation is simple and straightforward since the size of the data field is totally predictable.

With VARCHAR fields, you get a completely different story. For example VARCHAR(15) actually allocates dynamically up to 16 bytes, up to 15 for data and, at least, 1 additional byte to store the the length of the data. If you have the string 'hello' to store that will take 6 bytes, not 5. String manipulation must always perform some form of length checking in all cases.

The tradeoff is more evident when you do two things:
1. Storing millions or billions of rows
2. Indexing columns that are either CHAR or VARCHAR

TRADEOFF #1

Obviously, VARCHAR holds the advantage since variable-length data would produce smaller rows and, thus, smaller physical files.

TRADEOFF #2

Since CHAR fields require less string manipulation because of fixed field widths, index lookups against CHAR field are on average 20% faster than that of VARCHAR fields. This is not any conjecture on my part. The book MySQL Database Design and Tuning performed something marvelous on a MyISAM table to prove this. The example in the book did something like the following:

ALTER TABLE tblname ROW_FORMAT=FIXED;

This directive forces are VARCHARs to behave as CHARs. I did this at my previous job back in 2007 and took a 300GB table and sped up index lookups by 20%, without changing anything else. It worked as published. However, it did produce a table almost double in size, but that simply goes back to tradeoff #1.

You could analyze the data being stored to see what MySQL recommends for column definition. Just run the following against any table:

SELECT * FROM tblname PROCEDURE ANALYSE();

This will traverse the entire table and recommend column definitions for every column based on the data it contains, the minimum field values, maximum field values, and so forth. Sometimes, you just have to use common sense with planning CHAR vs VARCHAR. Here is a good example:

If you are storing IP addresses, the mask for such a column is at most 15 characters (xxx.xxx.xxx.xxx). I would jump right at CHAR(15) in a heartbeat because the lengths of IP addresses will not vary all that much and the added complexity of string manipulation controlled by an additional byte. You could still do a PROCEDURE ANALYSE() against such a column. It may even recommend VARCHAR. My money would still be on CHAR over VARCHAR in this instance.

CHAR vs VARCHAR issues can be resolved only through proper planning. With great power comes great responsibility (cliche but true)

Answer 2

Most of the answers in this thread are over five, twelve years old, written before InnoDB and utf8 were defaults. So, let me start over...

When a query needs an internal temporary table it tries to use a MEMORY table. But MEMORY cannot be used if

• TEXT/BLOB columns being fetched, even TINYTEXT.
• VARCHAR bigger than some amount, probably 512 in the current version.

Also, note that VARCHARs are turned into CHARs. (8.0 modifies this.) So, VARCHAR(255) with a CHARACTER SET utf8 expands to 765 bytes, regardless of what is in the column. Then, this might be triggered:

• If the MEMORY table gets bigger than either max_heap_table_size or tmp_table_size, it will be converted to MyISAM and potentially spill to disk.

So, VARCHAR(25) is more likely to stay MEMORY, hence be faster. (255) is not as good, and (64000) is bad.

(In the future, temp tables will probably be InnoDB, and part of this answer will need revising.)

(Update) MySQL 8.0.2: "The TempTable storage engine replaces the MEMORY storage engine as the default engine for in-memory internal temporary tables. The TempTable storage engine provides efficient storage for VARCHAR and VARBINARY columns." (Since then, there have been further changes to the handling of temp tables; I suspect the dust has not settled yet.)

Answer 3

The answer to this is actually rather complex. The short version: there is a difference.

1. When creating temporary tables to filter results (e.g. GROUP BY statements), the full length will be allocated.

2. The wire protocol (sending rows to the client) will likely allocate the larger length.

3. The storage engine may/may not implement a proper varchar.

For (2) I admit the wire protocol is not something I am intimately familiar with, but the general advice here is try and apply at least some minimal effort to guess the length.

Answer 4

It's my understanding that the smaller fields may be includable in the index directly, whereas the longer ones cannot. Due to that limitation, if you want the strings to be indexable, I would say keep them shorter. Otherwise, no, being as how they're both varchar then ops like sorting or comparing will operate in like time, whether the fields are 25 or MAX.

Answer 5

A varchar column that size makes queries on the entire table more likely to use temporary tables. According to the High Performance MySQL book. When the optimizer tries to see if it can run this query in memory or if it needs a temp table, it looks at the row size based on the table definition, meaning, for speed it does not try to see how much of the 64K characters you re actually using. This is why the writers recommend you not stretch out that definition way beyond the actual possible values that would go in the column. Obviously, if you set yourself up for more queries going into temp tables (even if the actual data size could fit in RAM) you have now incurred I/O penalties you could have avoided.

Answer 6

ensure you don't run out of room

This phrase implies that you ask the question because you're not sure about the data you'll be storing in the database. If that's true, you'll be well served to find out as soon as you can, because you'll need that for capacity planning. If you might be getting data elements with 7000 characters, for example, you need to know because that would have performance implications on any DBMS.

That said, I prefer to have column sizes related to the expected contents. For example, a phone number is unlikely to be longer than 50 characters, even if you include a country code and extension. Similarly, a zip or postal code will most likely be 20 characters or less.