Skip to main content

Questions tagged [encoding]

Encoding is a set of rules used to represent data in a form that can be stored and transmitted to another process or system. Character encoding (e.g. Windows-1252, ISO-8859-1, UTF-8, UTF-16) refers to the way character data is represented as a series of bytes. Binary encoding (e.g. Base64) refers to the way binary data is transformed into a series of characters.

22 questions with no upvoted or accepted answers
Filter by
Sorted by
Tagged with
3 votes
0 answers
2k views

Converting a SQL_ASCII database to UTF-8

I have a postgresql (version 8.1) database that has the encoding set to SQL_ASCII, combined with several apps that some are running using UTF8 and some with LATIN1. With this scope I have a database ...
vsmoraes's user avatar
2 votes
1 answer
562 views

How can I detect double-encoded MySQL columns and rows, and validate the repair?

My database provider, bless their hearts, migrated our MySQL databases to another server recently, and introduced double-encoding of UTF-8 data via Latin1 into our text data. Strings like 'emdash—here'...
Jim DeLaHunt's user avatar
2 votes
1 answer
2k views

What is the impact of converting latin1/latin1_swedish_ci to utf8mb4/utf8mb4_unicode_ci?

I was facing some issues with the character's encoding. Those are resolved by updating the CHARACTER and COLLATE for some columns in the table. So my concern is Is this conversion safe? Or can this ...
Mehar's user avatar
  • 121
2 votes
1 answer
391 views

Unable to enter characters in MariaDB

I have just switched from MySQL to MariaDB and am running into a very silly problem: I cannot enter any extended characters in the database (for example, ö ä or å). The system is utf8, and I tried ...
jorgon's user avatar
  • 21
2 votes
0 answers
149 views

Create/Edit SSIS Derived Column Task From Text/Code for Large ETL

I have to insert a large UTF-8 encoded flat file to a 1252 encoded sql server. In order to do so, I am using SSIS to type cast each column. Is there a way, other than copy pasting through all 400 ...
Tyler Raber's user avatar
2 votes
0 answers
961 views

Fixing a UTF8 encoded database that was dumped using an incorrect encoding?

The question is probably a little vague so let me elucidate: System OS is Ubuntu Linux 14.10 LTS The Servers default encoding is Latin1 The database default encoding is/was UTF8 The data was ...
Narrim's user avatar
  • 121
2 votes
0 answers
1k views

Convert large utf8 database to utf8mb4

I have a large database (~450gb) which I need to convert from utf8 to utf8mb4. I'm aware of the steps I need to take, but is this practical on tables containing this much data? I'm mainly concerned ...
Jonathan's user avatar
  • 188
2 votes
0 answers
1k views

Postgresql Strange Characters

I've found a lot of strange characters on data rows like this ÃÃÂà The data inserted by perl script which converting to UTF8 ...
Hendra Budiawan's user avatar
2 votes
0 answers
1k views

Umlauts in DBeaver

I have a Postgres Database set to UTF-8. I am working with DBeaver 3.1.4, and it seems to use a different encoding than UTF-8 (I suppose Latin-1 or something), which leads to 1) wrong display in the ...
kutschkem's user avatar
  • 123
1 vote
0 answers
52 views

Issue with hash storage resulting from password_verify() output

Consider the following code: $random_token = random_bytes(32); $token_hash = password_hash( $random_token, PASSWORD_DEFAULT ); $token_base64 = sodium_bin2base64( $random_token, ...
DevelJoe's user avatar
  • 163
1 vote
1 answer
1k views

How to display non-English character correctly in db2cmd?

On Db2 v11.5.7 on Linux/x86_64 I have a UTF-8 database. Executing db2 get db cfg for test1 returns: Database code page = 1208 Database code set = UTF-8 On my Windows 10 computer in ...
folow's user avatar
  • 347
1 vote
0 answers
144 views

Data in one or more tables corrupt after recovering tables with alter table discard/import tablespace

I accidentally dropped my schema and had to recover all the tables per the steps in this post; Recover schema and the data in it from accidentally deleted schema However, I notice that all of the ...
Citizen1138x's user avatar
1 vote
0 answers
485 views

Convert any string to url valid percent encoding in BigQuery

I am trying to convert any string with any set of special characters into a valid url of the format below. In Bigquery Example: /artwork-v2/-̴̕ι-̶͔͛n̴e̷p̸u̴̒n̵uś̵̥o̵̙̾rt̷͗um̶̹͐-20380 encodes to: /...
Keegan Ead's user avatar
1 vote
0 answers
1k views

PSQL console encoding

How can I have a working PSQL console using UTF8 encoding under Windows? I have a Windows server and client. The Postgres 12 database contains tables with content in multiple languages (ex: English, ...
JGH's user avatar
  • 111
1 vote
0 answers
326 views

MariaDB REGEXP_REPLACE Invalid utf8 byte sequence

I'm working with a fairly old (15 years) old database. It started as a MySQL3, was upgraded several times, the frontends had problems with proper encoding, the whole database was converted to utf8mb4 (...
mindhaq's user avatar
  • 111
1 vote
1 answer
2k views

How to configure DBeaver to save files without a BOM for UTF files?

I am looking for the equivalent setting as the PGADMIN setting Write BOM for UTF files which will cause saved files to be saved with/without the BOM. I've found settings in DBeaver when exporting ...
Jay Cummins's user avatar
1 vote
0 answers
106 views

Merge Replication Business Logic Handler Incorrect Encoding when connecting to SQL Server

We have a SQL Server Merge Replication configuration that uses a business logic handler to do conflict resolution on a datetimeoffset column. All we're doing is a copy of the Subscriber or Publisher ...
Jason's user avatar
  • 66
0 votes
0 answers
120 views

Redshift: Any difference in performance between # and temporary tables?

Is there any difference between the below two styles of using temporary tables in Redshift -- in terms of cluster performance (or) R/W into and from the table? I am new to Redshift, hence bit unaware ...
ranit.b's user avatar
  • 131
0 votes
0 answers
60 views

Create dadabase with spesific encoding from shell script

I want to create a database from shell script, I add this line to my script psql -c 'CREATE DATABASE template1 WITH OWNER = postgres ENCODING = 'LATIN1' TABLESPACE = pg_default LC_COLLATE = "...
user3093583's user avatar
0 votes
2 answers
391 views

How mysql store data and their encoding

I have few databases in latin1 and i will migrate it to utf8. I have some characters like 'œ' that are utf8 characters. I want to know how mysql store the encoding ? Cause if it stores utf8 characters ...
Thomas Corbisier's user avatar
0 votes
1 answer
735 views

How to get line that causes error when loading from csv to MySQL

I'm trying to load data from CSV file, but occurs error Error Code: 1366. Incorrect integer value: 'abcdxxx' for column 'col' at row 123545. I Have file with multiple breaklines CRLF's in content, ...
R. Nec's user avatar
  • 159
-1 votes
1 answer
1k views

UTF-8: showing correct in database, but not in webpage

When i upload data from php post forums "हिन्दी" language, changed into "हरिà" in database but looking correct in webpage, and when i upload data through phpmyAdmin from csv file "हिन्दी" ...
Praveen Yadav's user avatar