I have a C++ application that connects to SQL Server 2008 through FreeTDS unixODBC on an RHEL box.

However, i am unable to insert supplementary characters to the database through FreeTDS unixODBC.

[FreeTDS][SQL Server]Error converting characters into server's character set. Some character(s) could not be converted

This is the query that my application is hitting

INSERT INTO mytable (SAMPLE) VALUES(N'乕乭𠁄𠁉乺丕')

== version details ==
unixODBC version - 2.3.0
freeTDS version - 0.95

I can successfully insert the characters in BMP. This characters are inserted perfectly fine. INSERT INTO mytable (SAMPLE) VALUES(N'乕乭乺丕')

While using Microsoft SQL Server Native Client for Linux, the supplementary characters are inserted properly.

== freetds conf ==
host = mysever.mydomain.com
port = 1433
tds version = 7.3
debug flags = 0xffff
dump file = /tmp/dump.log
dump file append = no
client charset = UTF-16
use utf-16 = yes

unixODBC v2.3.0 has been built with

./configure --prefix=/usr/local/ --enable-iconv --with-iconv-char-enc=UTF8 --with-iconv-ucode-enc=UTF16LE

FreeTDS v0.95 has been built with

./configure --prefix=/usr/local --sysconfdir=/usr/local/etc --with-tdsver=7.3

  • Have you set the use utf-16 option to yes in the freetds.conf file? The default is no (i.e. use UCS-2, which doesn't know how to interpret the surrogate pairs). Feb 28 '16 at 7:34
  • @srutzky, updated my post based on ur settings recommendation. still the same issue.
    – D3XT3R
    Feb 29 '16 at 9:03
  • Hey there. Did you ever make any progress on this? If so, what did you do? If not, are you still wanting help? I last suggested (in my answer) to try both upgrading unixODBC to a more recent version, and to try client charset = UTF-8 in the freetds.conf file. Else it would help to see what info is in the /tmp/dump.log file. Nov 6 '16 at 1:41

Things to look at / try:

  1. Looking at the documentation again (now that you have posted the config file and configuration options), I think your setting of client charset is incorrect. You specify --with-iconv-char-enc=UTF-8 when configuring the build for the ODBC driver, so I suspect you need to also use UTF-8 when specifying the client charset option (or whatever encoding your app code is using).

  2. Try rebuilding FreeTDS with --enable-odbc-wide-tests. This might help narrow down where the issue is (assuming the test uses Supplementary Characters).

  3. Did you see the note about "Version 0.95 removed support for iconv" on the FreeTDS page for How to build: Configure and make? Not sure what, if any, impact this has, but if nothing else helps then you need to verify this part (i.e. that your build of iconv works and that it is being used).

  4. Looking at the unixODBC website, the main page has a version history and you claim to be using 2.3.0 which, according to that history, was released in April 20th, 2010 (almost 6 years ago). Since then there have been several "wide character"-related bug fixes, as well as many other fixes. Even if some other change (#1 perhaps) fixes the immediate issue, I would still upgrade to the most recent version of this driver, 2.3.4, which was released on August 31st, 2015.

  5. Something to consider, perhaps, but if the Microsoft driver does work, why not just switch to using that?

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.