We’re rewarding the question askers & reputations are being recalculated! Read more.

Questions tagged [character-set]

The tag has no usage guidance.

Filter by
Sorted by
Tagged with
17
votes
4answers
8k views

Why does the varchar datatype allow unicode values?

I have a table with a varchar column. It is allowing Trademark(™), copyright(©) and other Unicode characters as shown below. Create table VarcharUnicodeCheck ( col1 varchar(100) ) insert into ...
15
votes
2answers
32k views

How can I insert smiley faces into MySQL ( 😊 )

I'm on MySQL 5.5.21, and trying to insert the '\xF0\x9F\x98\x8A' smiley face character. But for the life of me, I can't figure out how to do it. According to various forums which I've been reading, ...
13
votes
3answers
26k views

Modify all tables in a database with a single command

Is there a single or a one line command to modify all tables within a database. I would like to issue this command in every table within a database: ALTER TABLE `table_name` CONVERT TO CHARACTER SET ...
12
votes
1answer
3k views

Error starting SQL Server 2017 service. Error Code 3417

I have SQL Server 2017 installed on my computer. This is what SELECT @@VERSION returns: Microsoft SQL Server 2017 (RTM-GDR) (KB4293803) - 14.0.2002.14 (X64) Jul 21 2018 07:47:45 Copyright (C) ...
12
votes
1answer
42k views

Why default character_set_server is latin1?

I am using MySQL 5.5 and when I show variables about charset, I have +--------------------------+----------------------------+ | Variable_name | Value | +--------------...
8
votes
2answers
47k views

Determine Oracle session client character set?

I know how the database characterset (NLS_CHARACTERSET in select * from v$nls_parameters;) and the client character set (the client environment setting NLS_LANG) interact. What I can't find out ...
7
votes
2answers
9k views

How to convert control characters in MySQL from latin1 to UTF-8?

While converting a database to UTF-8 I noticed a strange behavior regarding the control characters 0x80-0x9F. For example, 0x92 (right apostrophe) would not get converted to UTF-8 and truncate the ...
6
votes
3answers
3k views

Why are there different levels of MySQL collation/charsets?

Looking at the MySQL server variables the server and database collation are set to latin1_swedish_ci but the collaction_connection is utf8_general_ci. Additionally, the collation/charset is set at ...
6
votes
2answers
2k views

What is the SQL Server equivalent of MySQL's utf8mb4 character set?

We have some web applications based on databases using utf8mb4 as character set and utf8mb4_Standard as collation. We saw that we can use any character we want with this setting. In SQL Server ...
5
votes
3answers
7k views

Is there any doc listing standard <=> Oracle character sets correspondencies?

I'm looking for a table listing Oracle and "standard" character sets, for example: Oracle Standard ------------ ---------- WEISO98859P1 iso-8859-1 AL32UTF8 utf-8 etc. etc. Does ...
5
votes
1answer
10k views

Do I need to use NVARCHAR rather than VARCHAR to properly store UTF8 characters in Oracle?

I want my Oracle database to properly store ALL possible UTF8 characters. If NLS_CHARSET is set to AL32UTF8 is that all I need to do? Or do I need to also make sure I use NCHAR and NVARCHAR rather ...
5
votes
1answer
3k views

Collations and codesets in PostgreSQL 9.x

In Windows 7, the collation is set to "English_United States.1252" when I select the "English, United States" locale in the installer. In Linux, the collation is set to "en_US.UTF-8" by default. I ...
5
votes
1answer
81 views

Query to find rows containing ASCII characters in a given range

I am using some scripts from another topic, but the accepted answer isn't working for all my data scenarios. I would have asked my question on the original How to check for Non-Ascii Characters post, ...
5
votes
1answer
2k views

How do I list all available encoding types available in postgreSQL?

SELECT datname, pg_encoding_to_char(encoding) FROM pg_database; …lists all the databases, each with its encoding type. However, I am trying to find out what are all the encoding types available in ...
5
votes
1answer
1k views

Dump with special characters in table column name

I'm taking over an old database, and while doing recovery tests from my dump I found out a table is wrongly backed up, the table definition is truncated. Looking at the live db I've found a column ...
5
votes
3answers
2k views

Querying special characters

I work with language with special characters (example: e -> é). I have a query with a LIKE but when I search a word with special char and my query doesn't have the special char, I don't get the ...
4
votes
1answer
13k views

Setting default charset/collation for MySQL database

I have a MySQL DB that used to be latin1, and I changed all the columns in all the tables to be utf8. However, I evidently didn't do all I had to do, because I just found that I have a crazy mixture ...
4
votes
1answer
5k views

What does WE8DEC mean?

It is one of the commonly used character sets. What does it stand for? Is it an acronym for something?
4
votes
2answers
1k views

MySQL - Changing DB CHARACTER SET

I have a 30GB MySql DB, made of innoDB tables. currently the tables character set is: "utf8_unicode_ci" and I want to change it to "utf8_general_ci", what the best way to do it? currently i'm going ...
4
votes
1answer
6k views

What's the differences between utf8_general_ci and utf8_unicode_ci and utf8_binary collation in MySQL?

I'm can't find the documentation of MySQL on this topic. Anyone can give some explanations please?
3
votes
1answer
6k views

MySql command line cyrillic Iinsert result is corrupted

Have putty connection. Putty config : window-->translation-->remote character set-->utf-8 Launch command line MySQL client. Perform insert with cyrillic like: CREATE TABLE `category` ( `catName` ...
3
votes
2answers
16k views

Encoding problems with sqlplus, Linux

I use sqlplus and all non-Latin characters appear as '?' when I run select queries. When I run insert or update queries with non-Latin characters I get question marks on a web page. select * from ...
3
votes
2answers
1k views

Are there any known problems caused by having NLS_LENGTH_SEMANTICS set to CHAR on an Oracle database?

I've always thought that anyone using a database where NLS_LENGTH_SEMANTICS set to BYTE when using a multi-byte character set needed their heads examined, but recently a customer of ours said that ...
3
votes
2answers
17k views

Tell MySQL to start using utf-8 encoding without `convert to`ing it

In a fairly unique situation, my team has ended up with UTF-8 bytes in a database that thinks the data is encoded as latin1. At least, I'm 85% certain that this is the situation at hand. For example,...
3
votes
2answers
3k views

Character Set Database Migration

This is a follow-up question to a previous thread I had posted. We're in the process of migrating to a new character set. Now, I am a programmer and we have DBA's who will be performing the work but I'...
3
votes
2answers
942 views

Change default collation for character set utf8mb4 to utf8mb4_unicode_ci

When creating a database without specifying a character set or collation the servers defaults are used (as expected). MariaDB [(none)]> SHOW VARIABLES LIKE '%_server' ; +----------------------+----...
3
votes
2answers
909 views

Character Set Encoding in a Table

I have the following data:Trollhättan If I print out and setting up the header to utf-8, then this is the output in the browser. If I'm not setting up the utf-8 in the header, Trollhättan. However,...
3
votes
4answers
58k views

Converting varbinary data to varchar in SQLServer leads to unexpected results

I need to fix some data conversion issue in SQL server 2008. I got some requirement change on storing the data. Without much care I converted existing varbinary data to varchar using CONVERT(NVARCHAR(...
2
votes
4answers
16k views

unable to insert utf8mb4 characters in mysql 5.6

recently i was facing problem in inserting emoji and other 4 bytes character in my mysql database then i found that my mysql version was 5.1 and it does not support utf8mb4 so i upgraded to mysql 5.6 ...
2
votes
1answer
14k views

How to disable case sensitivity in SQL Server 2012?

I tried to update collation_name in sys.databases. I am trying to change 'SQL_Latin1_General_CP1_CS_AS' to 'SQL_Latin1_General_CP1_CI_AS' in order to disable case sensitivity in table. update d set d....
2
votes
2answers
2k views

SQL Server imports data in wrong charset with SSIS

I have a flat file which I am importing into a SQL Server table using a SSIS package. The code page is set to 65001. However, when I run a SELECT * FROM tableName the data appears mangled. For ...
2
votes
2answers
617 views

Which accented characters are considered the same?

In MySQL, certain characters are considered the same when performing searches, e.g. 'n' and 'ñ'; unless you specify BINARY in the condition. Is there a list somewhere of the characters that are ...
2
votes
2answers
9k views

SQL Server to MySQL migration - how can I remove UCS-2 surrogate pairs in SQL Server?

I have been tasked with migrating a SQL Server 2005 database to MySQL 5.6 (these are both database servers runnig locally) and would really appreciate some help. -------Edit - Answered------- This ...
2
votes
1answer
17k views

€ Euro sign/symbol not showing up on Oracle 10gr2 and aix 6.1

this is Oracle 10gr2 on aix 6100-02 The issue is that I do not manage to see the € sign from Oracle. This is an easy way to see the issue: SQL> insert into prueba values ('€'); SQL> commit; ...
2
votes
2answers
7k views

Postgres client unable to detect UTF-8 database

Context I am deploying a Vagrant box for my development team, and have encountered a problem restoring UTF-8 collated dumps. I don't believe it is related to the now-infamous "UTF8" has no equivalent ...
2
votes
2answers
143 views

Using CONVERT TO CHARACTER SET for changing only collation?

I think the answer to this is an obvious yes, but I'm having trouble finding confirmation on it. All the questions related to charsets seems to be about changing the charset and not only collations. ...
2
votes
2answers
511 views

How do I use 'N' in my prepared SELECT statement

I'm connecting to an SQL Server 2008 database using pdo and the ODBC Driver 13 for SQL Server. I'm trying to run the following query which searches the database for Chinese characters: SELECT TOP (...
2
votes
2answers
9k views

MySQL silently replaces UTF chars with literal question marks

I'm experiencing a situation similar to this SO question i.e. I'm working with a legacy database which has UTF8 content in latin1 tables (pretty ugly I know). Now I'm getting new data from a new ...
2
votes
2answers
37k views

How to convert a MySQL database from utf8mb4 to utf8?

I would like to convert a MySQL database, which has utf8mb4 character set and utf8mb4_unicode_cicollation, to utf8 and utf8_unicode_ci. How to do that please? Thank you in advance for your ...
2
votes
2answers
4k views

How to convert html entities in Oracle?

I'm trying to decode html containing html entities. I've tried dbms_xmlgen.convert and utl_i18n.unescape_reference but the results are less than satisfactory. SET ESCAPE ON; SELECT 'dbms_xmlgen....
2
votes
1answer
1k views

Oracle 10 to 11: charset issue

I was redirected from SE's root site to this one. I'm not a DBA, so I shouldn't be doing this, but I have to, so I apologize beforehand to all of you DBAs that will read this post: I'm migrating an ...
2
votes
2answers
5k views

ORA-12899: importing a .dmp file + character set conversion

I'm working on a django project on Ubuntu 10.04 with Oracle Database Server , so I've installed: Oracle Database 10g xe universal Rel.10.2.0.1.0 cx_Oracle-5.0.4-10g-unicode-py26-1.x86_64 When I ...
2
votes
1answer
798 views

Correcting the imported latin1 data to be utf8

I restored my drupal site database from a backup but after i finished i discovered that the Imported data was symbols like this قوانين كرة because the backup was taken using latin1 charset ...
2
votes
1answer
2k views

“Data truncated” vs “Incorrect string value”

I'm trying to figure out the difference between some warning messages related to data truncation. Consider the following table: CREATE TABLE `txttest` ( `mycol` text NOT NULL ) ENGINE=...
2
votes
2answers
47 views

charindex thrown off by extended characters

I've got a column with a filenames stored in a nvarchar(255). I'm trying to parse out the file extension using reverse( left( reverse(filename), charindex('.', reverse(filename) ) -1 ) ) This works ...
2
votes
1answer
3k views

Converting character sets and foreign key constaints

I'm trying to convert the character set and collation of my MySQL database and all containing tables from utf8/utf8_unicode_ci to utf8mb4/utf8mb4_unicode_ci. I will refer to my database name as: MyDB ...
2
votes
2answers
3k views

UTF8 Trouble while migrating from MSSQL to MySQL with MySQL Workbench

I am tasked with migrating part of our database from an on-site MSSQL server to an Amazon RDS MySQL. This will be my first attempt with a database migration, so I am doing a dry run from our MSSQL ...
2
votes
2answers
3k views

Removing per column and table collation and charset

I recently completed a transition of a database from CHARSET=utf8 COLLATE=latin1 to CHARSET=utf8mb4 COLLATE=utf8mb4_unicode_ci. During the process tables were migrated one at a time by altering their ...
2
votes
1answer
76 views

Extract all unique chars from all texts across the table

I have a huge mysql table (20 columns by 320.000 rows) that contains a lot of text from a lot of languages and the problem is that I need to identify all the unique characters in that table. For ...
2
votes
1answer
2k views

Remove question marks from database(latin1_general_ci table)?

How can I remove question marks(not diamond shaped) from a latin1_general_ci MyISAM table? I tried going back to utf8 and tried to apply things in https://stackoverflow.com/questions/1476356/...