Starting in SQL Server 2019 (currently in beta / "Community Tech Preview"), there is native support for UTF-8 via a new series of UTF-8 collations. HOWEVER, having the ability to use UTF-8 does not mean that you should. There are definite drawbacks to using UTF-8, such as:
- Only the first 128 code points are 1 byte (i.e. the standard 7-bit ASCII set)
- The next almost 2000 code points are 2 bytes, hence no space savings over UTF-16 /
- The remaining 63k code points in the BMP (i.e. the U+0800 - U+FFFF range) are all 3 bytes, hence 1 byte larger than the same character in UTF-16 /
- Just have it stated: Supplementary Characters are 4 bytes in both encodings, so no space difference there
- While you might save space using UTF-8, there is a very good chance you will take a hit on performance for doing so.
What it really comes down to is this: UTF-8 is a storage format design to enable 8-bit systems (that were typically designed around ASCII and ASCII Extended -- Code Pages) to use Unicode without breaking anything or requiring any modification of existing files in order to keep things running. UTF-8 is wonderful for file systems and networking, but data stored inside SQL Server is neither. The fact that data that just happens to be mostly (or entirely) within the standard ASCII range requires less space than the same data when stored as UTF-16 /
NVARCHAR is a side-effect. Sure, it's a side-effect that can prove useful, but that decision needs to be made by someone who understands both the data and the consequences / drawbacks of this decision. This is not a feature for general use.
Also, the main use-case for UTF-8 (in SQL Server) is for app code already using UTF-8, possibly already with another RDBMS that supports it, and there is no desire or ability to update the app code / DB schema to use
NVARCHAR datatypes (for tables, variables, parameters,etc), or to prefix string literals with an uppercase "N". The goal is the same as the reason for UTF-8 existing: enable app code to use Unicode without changing the overall structure or rendering exist data invalid. If this describes your situation, then use UTF-8, but be aware that there are still a few bugs / issues with it.
If you do not have an explicit need for Unicode working without using
NVARCHAR or uppercase "N" prefixed string literals, then the only other scenario where UTF-8 is a benefit is if you have A LOT of mostly standard ASCII data that needs to allow for Unicode characters, and you are using
NVARCHAR(MAX) (which means that data compression won't work), and the table gets updated frequently (so Clustered Columnstore Index is probably not going to truly help).
For full details, please see my post:
Native UTF-8 Support in SQL Server 2019: Savior or False Prophet?