Skip to main content

Questions tagged [character-set]

The tag has no usage guidance.

Filter by
Sorted by
Tagged with
0 votes
0 answers
25 views

Why is the character set for the uca1400_ai_ci collation NULL?

I've been working with database collations of late. I generally don't like ambiguous queries such as SHOW COLLATION so I eventually found the source via the following query: SELECT * FROM ...
John's user avatar
  • 737
0 votes
0 answers
23 views

Mariadb 10.11 on Debian 12 and character_set_database

I have a stock mariadb instance on debian 12 with no edits to the config files. When I create a table w/o specifying character set and collation, the table is created with latin1 and it's collation. ...
user44021's user avatar
1 vote
0 answers
93 views

MySQL 8.0 fails to store 5.7-allowed symbols (Incorrect string value, Error 1366) [closed]

I'm upgrading MySQL 5.7 server to 8.0 with a bunch of user-input data that sometimes contain "malformed" unicode characters like x'eda0bdedb98f'. The latter seems to be 2 UTF-16 characters ...
Alessandro Cattaneo's user avatar
-1 votes
1 answer
52 views

What is the best collation for json data?

When converting my database to uftmb4 I noticed some columns were using utf8mb3_bin, and my command just altered them to utf8mb4_unicode_520_ci instead of utf8mb4_bin. Why ? And also which one is ...
Freedo's user avatar
  • 196
0 votes
0 answers
55 views

Mariadb can't enforce charset and collation to clients

I'm trying to force clients to use utf8mb4_unicode_520_ci and uft8mb4. I tried to add this to all my galera cluster nodes [mysqld] ... collation-server = utf8mb4_unicode_520_ci init-connect='SET NAMES ...
Freedo's user avatar
  • 196
0 votes
2 answers
30 views

List of characters managed by a collation

I'm migrating to MySQL 8.0.33. We are changing character set to utf8mb4. For collation, I'm wondering among different options. I'm looking for documentation to review the list of characters that are &...
yaki_nuka's user avatar
  • 141
0 votes
0 answers
112 views

Having the same charset and collation between Redshift and MySQL

I looking to migrate data between Redshift table and MySQL that has a string column. There are a few differences between each engine: Redshift is case sensitive, and by default MySQL isn't. MySQL, ...
Nir's user avatar
  • 533
0 votes
0 answers
107 views

Update character set for all columns in entire MySQL server

I'm upgrading from MySQL 5.7 to 8.0 and right now cleaning the old stuff. I used Upgrade Checker to verify my environment. Lot's of things are cleaned now, but one of the issues I have currently is ...
Marek's user avatar
  • 13
0 votes
1 answer
269 views

Joining Columns with different character set

I am joining two tables on varchar columns having CHARACTER SET utf8 COLLATE utf8_bin and one with CHARACTER SET latin1. Indexing is not working when i am joining these two tables on this column, I am ...
Manish Kumar's user avatar
0 votes
1 answer
233 views

Exporting via Phpmyadmin Charset

MySQL 5.7 I am exporting my db from my web host using phpmyadmin. This produces a single .sql file. When I check the export with Notepad+, there are special characters in the file. e.g. in my source ...
TheOrdinaryGeek's user avatar
1 vote
0 answers
48 views

How does mysql_real_escape_string() work?

I have to send some string (in C) as a query to MySQL, so i used mysql_real_escape_string() to escape some characters like \0 or \n: #include <mysql/mysql.h> int loginQuery(char *Nickname, char *...
SempriGno's user avatar
0 votes
1 answer
155 views

Interpretation of impdp/expdp character set warnings

impdp and expdp report encoding warnings in the form of: export done in A character set and B character set import done in C set and C character set Why do both warnings name two character sets? What ...
Basti's user avatar
  • 103
0 votes
1 answer
961 views

Tables character-set migration without huge downtime- mysql

We're in the process of migrating to a new character set for several tables in our database. Some tables are currently using utf8/utf8mb3 character set, which is deprecated now and we want move on ...
Retrosec6's user avatar
2 votes
1 answer
448 views

How can I detect failure in CONVERT( tbl AS utf8)?

MySQL provides the CONVERT(`tbl` AS utf8) expression to transcode text from one charset to another. Sometimes that conversion cannot succeed, because the destination charset does not include ...
Jim DeLaHunt's user avatar
2 votes
1 answer
576 views

How can I detect double-encoded MySQL columns and rows, and validate the repair?

My database provider, bless their hearts, migrated our MySQL databases to another server recently, and introduced double-encoding of UTF-8 data via Latin1 into our text data. Strings like 'emdash—here'...
Jim DeLaHunt's user avatar
0 votes
2 answers
905 views

What is the definition of latin1 character set in MySQL 8.0 and MySQL 5.7?

I am trying to fix some corruption to character fields in my database after a migration. I suspect that part of the problem is that fields with UTF-8 data were interpreted with the MySQL latin1 ...
Jim DeLaHunt's user avatar
0 votes
1 answer
44 views

Valid charset check

I need your help. One of our integrations is failing as we only certain charset What i would like to do is run an adhoc query on the following column (FirstName) and flag any record which is not a ...
Masond3's user avatar
  • 133
1 vote
1 answer
867 views

Unknown charset: utf8mb4

Context: I send queries to my MariaDB server(Ver 15.1 Distrib 10.3.34-MariaDB) through a Python script and mysql.connector module. (I don't know which one it's among those 3 in my list: mysql-...
lyeaf's user avatar
  • 177
0 votes
2 answers
823 views

How can you repair a MySQL 8.0 Installation after changing `character_set_system` from utfmb3 to utfmb4 fails?

NOTE: My issue was due to a syntax error in the my.cnf file, not anything to do with errors on converting to utf8mb4. I still do not have any idea how you do that. I posted an answer explaining what ...
shelleybutterfly's user avatar
4 votes
1 answer
14k views

Storing special characters (German, French, Spanish) in SQL Server result in weird strings

I'm trying to store the German ß in SQL Server, but it results in ß. For example "Gleimstraße" is stored as "Gleimstraße" But this also happens for other characters: "König-...
Adam's user avatar
  • 245
1 vote
1 answer
2k views

How to fix unicode REPLACEMENT CHARACTER � in postgres database

Is there a way fix the � problem in postgres database: As you can see there are names with accents and are not displayed corretly in the database. I would like to know if there is a way to fix this ...
rainman's user avatar
  • 195
2 votes
2 answers
529 views

Index Collision in UTF8 on MariaDB

I am having a problem moving (converting) a DB from SQLite3 to MariaDB 10.3.29 server running on a Synology NAS. One table, actor, has 3 columns actor_id name art_urls Some actors have 2 entries in ...
Windom Sparks's user avatar
1 vote
1 answer
268 views

Character set mismatch in database migration

I have an old database (MySQL 5.1.58) that I want to migrate to a new installation (MySQL 8.0.26). When I just use oldserver ~$ mysqldump mydatabase > file.sql newserver ~$ mysql mydatabase < ...
Bart Friederichs's user avatar
0 votes
0 answers
4k views

Set 'WE8ISO8859P15' character set for existing Oracle 19c DB while migrating 12c DB

I have a database in 12c (12.2.0.1) which I need to migrate to an existing live oracle 19c DB.(This DB already hosts a couple of live DBs’s) Character set of DB’s are as below: 12c -> ...
picku's user avatar
  • 1
1 vote
2 answers
629 views

Determine Ideal Collation Set for correct data storage

Alright so I have a MariaDB DB and I've just noticed some weird storage issues, like the following: My JSON columns (which in MariaDB are LONGTEXT) have an automatically set collation of utf8mb4_bin. ...
DevelJoe's user avatar
  • 163
0 votes
0 answers
216 views

MySQL character set CLI and Workbench show different results

Q: Character sets shown by the MySQL CLI command lines and my MySQL workbench show different results. Why is that? I've run into MySQL 'chcp' error and cannot view my server status after updating my ...
hansf14's user avatar
0 votes
0 answers
2k views

Convert VARCHAR2 to UTF-8 in Oracle 12

I need to convert VARCHAR2 columns that store text in unkown character sets into a standard UTF-8 BLOB. The table has more columns, but the ones of interest to me are: ADDRESS POST ---------- ----...
Joe DiNottra's user avatar
1 vote
1 answer
762 views

MySQL query show something like 0x8081 instead of special characters

When i run SELECT CHAR(128,129,130,131,132,133,134,135,136,137); I got 0x80818283848586878889 istead of Çüéâäàåçêë. Does anybody know why?, I'm using charset utf8mb4. When I run show variables like '%...
Danh Le's user avatar
  • 11
2 votes
1 answer
2k views

What is the impact of converting latin1/latin1_swedish_ci to utf8mb4/utf8mb4_unicode_ci?

I was facing some issues with the character's encoding. Those are resolved by updating the CHARACTER and COLLATE for some columns in the table. So my concern is Is this conversion safe? Or can this ...
Mehar's user avatar
  • 121
8 votes
2 answers
13k views

Best General Purpose Character Set and Collation for MySQL

Currently, whenever I create a new MySQL database, I use utf8mb4 as a character set and utf8mb4_unicode_520_ci for the collation, e.g.: CREATE DATABASE IF NOT EXISTS db_name DEFAULT CHARACTER SET ...
GTS Joe's user avatar
  • 253
2 votes
1 answer
411 views

Unable to enter characters in MariaDB

I have just switched from MySQL to MariaDB and am running into a very silly problem: I cannot enter any extended characters in the database (for example, ö ä or å). The system is utf8, and I tried ...
jorgon's user avatar
  • 21
0 votes
1 answer
8k views

Why do I get ERROR 1824 (HY000) while importing MySQL dump from 5.7 to 8.0.x?

I have a MySQL 5.7.x dump with about 12GB that I try to import into a newly installed MySQL 8.0.x local instance. mysql -u username -p test < DB-210508.sql The import starts but fails with the ...
merlin's user avatar
  • 313
1 vote
1 answer
2k views

MySQL illegal mix of collations, ASCII to UTF-8

I have a MySQL 8.0 table like this, which is UTF-8 apart from one field, which holds an ASCII UUID that doesn't need UTF-8 overhead: CREATE TABLE `things` ( `id` bigint unsigned NOT NULL ...
Synchro's user avatar
  • 261
1 vote
1 answer
54 views

Is CHARSET important for tables with INT only?

My server default CHARSET is UTF-8 (set in my.cnf). I create tables like CREATE TABLE t1 ( ID int(11) unsigned NOT NULL AUTO_INCREMENT, Title varchar(255), PRIMARY KEY(ID) ) ENGINE=InnoDB DEFAULT ...
Googlebot's user avatar
  • 4,531
0 votes
1 answer
310 views

Error: Unknown character set: 'ascii'

When trying to import a database from a .mysql file, I get an error: ERROR 1115 (42000) at line 24: Unknown character set: 'ascii' The part the error occurs at is from columns using ASCII: `column` ...
mbomb007's user avatar
  • 103
1 vote
1 answer
2k views

mysqldump dumps different data with and without --no-create-info

mysqldump dumps different representations of data when called with/without --no-create-info. Test case First, create the test table and populate it with interesting data. CREATE TABLE `test` ( `...
matt's user avatar
  • 203
0 votes
1 answer
33 views

MySQL - can't see end of

I am trying to pick the MySQL charset combination which is correct. Unfortunately, at least on my screen, the list of options is very narrow and doesn't show the full text of the options. I want to ...
Statsanalyst's user avatar
1 vote
1 answer
1k views

Default value for CHARSET when creating a table

I came across this query in a book i'm learning from. CREATE TABLE `articles` ( `title` varchar(1000), `content` text, `links` text, `image` varchar(500), `url` ...
Bucephalus's user avatar
4 votes
2 answers
1k views

Does MySQL 8 ASCII vs utf8mb4_0900_ai_ci size differ when only using ASCII characters?

If I only use only ASCII characters, will VARCHAR (255) with utf8mb4_0900_ai_ci be larger on disk than VARCHAR (255) using ASCII?
jsHate's user avatar
  • 177
4 votes
1 answer
5k views

MySQL - Joining table with different charset in a view

I want to join tables from two different schemas. But the problem is both schemas follow different charset utf8_general_ci & utf8mb4_unicode_ci. When i do a join on both tables, it works. But, ...
Prateek Pande's user avatar
3 votes
2 answers
2k views

How to convert xml entities to unicode characters (with read-only access) in Oracle?

I'm connected to an Oracle Database (11g Release 2 - 11.2.0.4), with read-only access. Into this database, some of the data is uploaded via, or rather as, XML - and quite a few entries contain ...
nutty about natty's user avatar
1 vote
2 answers
1k views

Changing from utf8 to utf8mb4

MySql database at the moment has: Character Set: utf8 Default Collation Name: utf8_unicode_ci I want to change it to: utf8mb4 utf8mb4_unicode_ci I would also do the same thing for all tables in ...
Vicky Dallas's user avatar
0 votes
1 answer
1k views

Oracle 19c Installer Can't Select National Character Set

I am trying to install a new 19c instance of Oracle database in order to migrate a 12c database there. The 12c database uses a database character set of WE8ISO8859P1 and a national character set of ...
Luke's user avatar
  • 349
0 votes
1 answer
809 views

Confusion about charset for connected tables from MS Access

I run 10.4.12-MariaDB on Windows10 and I use 64-bit ODBC driver to connect tables from MS Access to MariaDB database. I am simply glad that it works (!). But going deeper and working more with ...
radimmachu's user avatar
0 votes
1 answer
1k views

Can I use impdp FULL=Y for an Oracle database charset conversion?

I need create a copy of an Oracle 12.2 instance (Oracle Restart) running on an Linux VM. The new instance needs a charset conversion from WE8ISO8859P1 to UTF8 I started by creating a clone of the VM ...
Max's user avatar
  • 195
0 votes
1 answer
72 views

Mysql utf-8 behaves differently into different environments

I have MySQL server under docker in 2 different environments: This is my create database: CREATE DATABASE guests CHARACTER SET utf8mb4 COLLATE UTF8MB4_UNICODE_CI; This is the create table: USE ...
SexyMF's user avatar
  • 85
1 vote
1 answer
798 views

converting latin to utf8mb4 causes questionmarks

The original format of the data is unknown The new table is in utf8mb4_general_ci If I do CONVERT(BINARY CONVERT(column USING latin1) USING UTF8) as mentioned here - it fixes all text, but converts ...
stackbox's user avatar
0 votes
1 answer
1k views

Question marks instead Hebrew characters after mysql dump and import

I tried to migrate my AWS RDS instance from Aurora to MySQL. I created a dump file from the Aurora and imported it into the MySQL instance. Both instances have the same characters set. mysql> ...
Omri's user avatar
  • 353
2 votes
2 answers
3k views

How to force mysqldump to produce /*!40101 SET character_set_client = utf8mb4 */;

I am attempting to dump my database, which is set to, and only contains, utf8mb4 and appropriate collations. I use the following line: mysqldump --add-drop-table --add-locks --allow-keywords --...
Torque's user avatar
  • 123
0 votes
0 answers
608 views

Will MariaDB treat ut8/utf8mb3 and utf8mb4 collations as compatible?

Background on relevant character sets MySQL and MariaDB support the deprecated utf8mb3 character set and collations. (utf8 is currently an alias for utf8mb3.) This supports all Unicode code points on ...
William Entriken's user avatar