We’re rewarding the question askers & reputations are being recalculated! Read more.

Questions tagged [encoding]

The tag has no usage guidance.

Filter by
Sorted by
Tagged with
3
votes
2answers
5k views

Not displaying special characters imported from a file

The scenario is a SQL Server instance, a database which is data fed using BULK INSERT operations mainly, and some of the texts inserted contain special characters like ñ because I am working in a ...
1
vote
0answers
368 views

Fixing a UTF8 encoded database that was dumped using an incorrect encoding?

The question is probably a little vague so let me elucidate: System OS is Ubuntu Linux 14.10 LTS The Servers default encoding is Latin1 The database default encoding is/was UTF8 The data was ...
4
votes
1answer
6k views

Equivalent of UTF8_UNICODE_CI collation in PostgreSQL

I would like a column in a table inside a PostgreSQL database (I am using version 9.6). I know of the UTF8_UNICODE_CI collation on MySQL, so I tried: CREATE TABLE thing ( id BIGINT PRIMARY KEY ...
0
votes
1answer
130 views

psqlODBC access encoding problem

My local machine: Windows 7, 32-bit. I have installed on it the psqlODBC for 32-bit. MS Access is 32-bit as well. I have tried both Unicode and Ansi connections to... The server: a 64-bit Ubuntu, and ...
1
vote
2answers
432 views

Mysql - latin1 (with latin2 content) to utf8

I have a database (ubbthreads) encoded in latin1 with content from latin2 (polish characters). Since latin2 is compatible with latin1 it looks fine on the website, however I cannot convert it in any ...
4
votes
2answers
7k views

Nvarchar to varchar to nvarchar characters conversion

I had a table filled with some greek text rows as nvarchar(2000). Lately, I altered the type of the column to varchar(4000) but I realized that some greek characters were shown as question marks. ...
2
votes
0answers
709 views

Convert large utf8 database to utf8mb4

I have a large database (~450gb) which I need to convert from utf8 to utf8mb4. I'm aware of the steps I need to take, but is this practical on tables containing this much data? I'm mainly concerned ...
1
vote
3answers
373 views

Fix double-encoded data

I have a field defined as Question VARCHAR(1024) CHARACTER SET utf8 NOT NULL. Our application stored double-encoded data into this field. For example, é shows up as é in phpMyAdmin, and C383C2A9 when ...
1
vote
2answers
4k views

CMD prompt can't display Unicode characters that are stored in oracle database correctly

I've been meddling with charset/encoding problems of Oracle database, PL/SQL developer and JDBC and these are what I want to achieve: Input Unicode characters from a browser into the database Display ...
2
votes
1answer
206 views

Tablediff utility output file encoding

When the SQL Server tablediff utility generates a file with differences between source and destination, if SSIS executes the script file which contains Arabic letters, then those letters are inserted ...
2
votes
1answer
995 views

Strange Garbage Character in SQL server database like ÂÃÂÃÂÃÂÃÂÃÂÃÂÃ [closed]

I am using HTMLBOX editor for getting Notes/RichText Data input from User. I found there are lot of garbage characters in my database and they keep on doubling on each time user save/update notes. ...
22
votes
1answer
28k views

How do I set a SQL Server Unicode / NVARCHAR string to an emoji or Supplementary Character?

I want to set a Unicode string variable to particular character based on its Unicode code point. I want to use a code point beyond 65535, but the SQL Server 2008 R2 database has a collation of ...
3
votes
3answers
552 views

TSQL Script Execution Through Power shell, Special Characters Rendering Problem

We have a simple T-SQL script file (.sql), which we execute through Powershell. Everything works fine, except the script portion, where we are using some special characters as string (example: 'ô' ). ...
1
vote
3answers
1k views

character set, not set correctly

I have two database with a table with utf8 table, I wanted to change the encoding of a column, I used two scenarios but a problem happened to me. at first the table was like this : CREATE TABLE `...
2
votes
1answer
4k views

ERROR 1071 (42000): Specified key was too long; max key length is 767 bytes

I have a databases with latin1 charachter set, I need to change tables character set to utf8, but I get error. the current character set is : mysql> show variables like "%character%"; +-----------...
0
votes
2answers
79 views

Convert all the data of table (which have both encoding data) into single encoding either utf8 or latin1 [closed]

We have table which consists of two different encoding data (utf8 and latin1) which has been inserted from two different use cases of application. We are getting broken strings issue for other ...
2
votes
0answers
548 views

Postgresql Strange Characters

I've found a lot of strange characters on data rows like this ÃÃÂà The data inserted by perl script which converting to UTF8 ...
1
vote
1answer
184 views

Can't search for non-ascii characters

I have a database with Turkish names and surnames. Here is some information about my table: CREATE TABLE `sorted_tc` ( `TCKIMLIKNO` double NOT NULL, `ADI` varchar(40) COLLATE utf8_unicode_ci NOT ...
1
vote
1answer
118 views

Move data from mysql 4.0 to 5.6 without corrupting encoding

I have a old mysql database that I would like to move to a recent version of mysql and optimize for SELECT ... WHERE ... statements. It is going to be a SELECT only database. Original database uses ...
4
votes
1answer
3k views

Postgres database encoding problem

I'm striving to convert badly encoded data from my table. For instance, I have a field with Nadège which should be Nadège. I tried using Postgres's functions convert, convert_from, convert_to ...
0
votes
0answers
224 views

GoldenGate: How to encode Oracle RAW columns as hex strings?

I am using GoldenGate to extract an Oracle table that contains RAW data. The destination is a flat CSV file. How can I encode the RAW columns to hex strings so that they can be written to the CSV ...
5
votes
1answer
8k views

Convert MySQL database from latin1 to utf8mb4 - and take care of German umlauts

I am using 5.6.28 MySQL Community Server to host WordPress + ProPhoto blog in German language at CentOS 6.7 Linux server: mysql> show tables; +-----------------------+ | Tables_in_blog | +--...
3
votes
1answer
7k views

“String or binary data would be truncated” error copying nvarchar(10) to char(10)

I am inserting values from one SQL table / column into another. The datatypes are different for these columns due to other reasons but I do not understand why nvarchar(10) source and char(10) ...
2
votes
1answer
400 views

mysqldump shows pairs of utf8 chars when dumping a utf8 database

I'll describe right away the problem, and put details at the bottom. I am dumping a database which I am fairly sure contains (almost: a few cells have wrong, latin1 encoded data) only UTF8 data. Also ...
2
votes
1answer
2k views

psql “invalid client_encoding” error on OS X, PostgreSQL 9.4.5

I'm currently running a OS X Lion Server system which ships with a built-in and not-upgradable PostgreSQL version. After years of usage I've finnaly decided to leave the built-in version and install ...
5
votes
1answer
1k views

Why does MS SQL Server return a result for empty string check when Unicode string is not empty

select * from (select N'ግዜ ' as t) as t2 where t= '' the string 'ግዜ ' matches the above check, why is this?
4
votes
1answer
226 views

How is encoding done in SQL database tables for pattern matching?

Are strings in table columns represented as bit patterns or Unicode? Does the database engine bring them into memory in a different representation character set from the disk-persisted ...
3
votes
1answer
2k views

How to change database encoding in an Amazon RDS Postgresql instance?

I need to create a database with SQL_ASCII encoding in Amazon RDS PostgreSql database instance. Since template0 and template1 are both with utf8 encoding, I get the following error $ createdb -h ...
0
votes
1answer
52 views

MySQL Cluster node utf8

I have configured MySQL Cluster (5.6) with two data nodes and managment node. Problem is that I cannot put special chars - looks like encoding is not set to utf8/utf16. I logged into each node and ...
0
votes
1answer
707 views

Issue with accented characters in SQLiteStudio

I downloaded a SQLite 3 dump of Northwind, and opened it in SQLiteStudio. Accented characters aren't displayed nicely. How to solve the issue? I suspect some encoding issue.
0
votes
1answer
493 views

How to get line that causes error when loading from csv to MySQL

I'm trying to load data from CSV file, but occurs error Error Code: 1366. Incorrect integer value: 'abcdxxx' for column 'col' at row 123545. I Have file with multiple breaklines CRLF's in content, ...
0
votes
1answer
231 views

How to determine character encoding in Hyper SQL database

I have Hyper SQL database and I don't know what is current character set. Is it set for whole DB or it can be different for every table? How to evaluate encoding in my base?
2
votes
2answers
9k views

MySQL silently replaces UTF chars with literal question marks

I'm experiencing a situation similar to this SO question i.e. I'm working with a legacy database which has UTF8 content in latin1 tables (pretty ugly I know). Now I'm getting new data from a new ...
2
votes
0answers
762 views

Umlauts in DBeaver

I have a postgres Database set to UTF-8. I am working with DBeaver 3.1.4, and it seems to use a different encoding than UTF-8 (I suppose Latin-1 or something), which leads to 1) wrong display in the ...
17
votes
4answers
1k views

How To Strip Hebrew Accent Marks

I need a Char Encoding Trick to Strip Hebrew Accent Marks. Sample Before בְּרֵאשִׁ֖ית בָּרָ֣א אֱלֹהִ֑ים אֵ֥ת הַשָּׁמַ֖יִם וְאֵ֥ת הָאָֽרֶץ Sample After בראשית ברא אלהים את השמים ואת הארץ
8
votes
2answers
3k views

Automatic Translation when Converting Unicode to non-Unicode / NVARCHAR to VARCHAR

Unicode code point 9619 is a character called "Dark shade": ▓ (http://unicode-table.com/en/search/?q=9619). Using the SQL_Latin1_General_CP1_CI_AS collation and 1252 code page, I would expect that ...
8
votes
1answer
25k views

Postgresql: set default psql client encoding

When I connect to a Postgresql DB using psql, I often get these messages: => SELECT * FROM question_view ; ERROR: character with byte sequence 0xd7 0x9e in encoding "UTF8" has no equivalent in ...
0
votes
2answers
1k views

Thai and English in a postgresql

I have a postgresql DB with Thai and English values/strings in several tables. Currently I have the following settings | Encoding | Collate | Ctype | | UTF8 | en_US.UTF-8 | en_US.UTF-8 ...
3
votes
0answers
2k views

Converting a SQL_ASCII database to UTF-8

I have a postgresql (version 8.1) database that has the encoding set to SQL_ASCII, combined with several apps that some are running using UTF8 and some with LATIN1. With this scope I have a database ...
0
votes
1answer
775 views

How to change the output encoding of Oracle command line tools to UTF8?

I am using a german oracle 11.2.0g install, which has some command line tools communicating with some non-ascii characters (ü, ö, etc.). These are written out in iso8859-1 encoding, but I need them as ...
2
votes
2answers
7k views

Postgres client unable to detect UTF-8 database

Context I am deploying a Vagrant box for my development team, and have encountered a problem restoring UTF-8 collated dumps. I don't believe it is related to the now-infamous "UTF8" has no equivalent ...
0
votes
1answer
2k views

PostgreSQL encoding conversion error

I have a Postgresql database in LATIN9 encoding. I recently migrated my web applications (PHP) from iso_8859_15 (latin9) to UTF8. So, now, customers are able to specify UTF-8 characters in web forms ...
13
votes
3answers
39k views

Using PostgreSQL 8.4, how to convert bytea to text value in postgres?

In my application I insert data in database using C code, since the strings I receive from an untrusted source I have escaped them using PQescapeByteaConn of libpq library. Which is working perfectly ...
1
vote
1answer
230 views

Which string variables in mysql support utf-8?

I have a table that stores string from a user and is displayed in a web form. But as I see when the user inputs e.g. in a language of cyrilic alphabet (I assume is UTF-8) garbage are displayed back. I ...
2
votes
1answer
3k views

How to install UTF8 locale on PostgreSQL?

I have a PostgreSQL 8.4 on Windows 7 machine, and my issue is that I have only the win1251 locale. So when I create a database, I choose the UTF8 encoding and can select only lc_collate = ...
1
vote
1answer
1k views

Client Encoding Mismatch when creating new database

I'm upgrading a program for work that uses postgres 9.2.4. When the program is run for the first time, it creates a production database, using the existing template database. The program used to run ...
3
votes
1answer
2k views

Confused over encoding/locale in postgresql

I am a little confused over the difference between encoding and locale as it pertains to the postgresql database. When initializing the database, you can specify both an encoding and a locale. ...
1
vote
1answer
131 views

Recover Wrong encoded string values in MySQL database

I have a database in which I store the alt and title values of the images. Unfortunately the program which inserts these values hasn't been connecting to the MySQL database with a UTF-8 link and the ...
2
votes
3answers
2k views

Restoring SQL_ASCII dumps to a UTF8-encoded database

I've got a Postgres 8.4 environment where the encoding on all our databases is set to SQL_ASCII - We're finally migrating to Postgres 9.2, and I would like to migrate everything over to UTF8 encoding. ...
19
votes
3answers
24k views

SQLCMD command is not able to insert accents

I am trying to run sqlcmd.exe in order to setup a new database from command line. I am using SQL SERVER Express 2012 on Windows 7 64 bits. Here's the command I use: SQLCMD -S .\MSSQLSERVER08 -V 17 -...