Randomly seeing database entries truncated to 253 characters on a php site and can't figure out why or if it could be malicious or a corruption issue. Any thoughts would be appreciated.
- Anybody can ask a question
- Anybody can answer
- The best answers are voted up and rise to the top
If not configured (by adding
My guess is that you've got a VARCHAR(255) column and at some point in the process, there's an inconsistency in character set configuration, and you have some UTF-8 characters that at some point in your system are being treated as latin1, causing the system to misinterpret the number of characters and truncate the value.
Wait, what? How is the number of "characters" different than the number of "bytes?"
In UTF-8 encoding, character values from 0x7F (decimal 127) downward are still encoded as their 1-byte ASCII equivalent. Larger values are encoded as 2 or 3 bytes. Common symbols like © (ASCII 169 0xA9) and ® (ASCII 174 0xAE) are encoded as two bytes, 0xC2 0xA9 and 0xC2 0xAE respectively.
When fully configured for UTF-8, MySQL understands this; using "®" as an example:
However... 0xC2 and 0xAE are both codes for valid characters in the default
In the example above, all of my character_set_* values (except
Now, I'll change these settings, to illustrate:
Note that I didn't actually change anything about my actual client's behavor -- each time the "®" symbol appears, the character was UTF-8 encoded and there were two bytes being sent and received over the wire. All that changed is that I only told MySQL that what I was sending and what I expected to receive should be interpreted as being sent to the server (