Questions tagged [encoding]

The tag has no usage guidance.

Filter by
Sorted by
Tagged with
-1
votes
0answers
20 views

Data showing correct in database, but not in webpage [on hold]

I have Indian languages in my database. When i check them through phpmyadmin they are shown perfectly. But in my website i can see only ?????. SHOW VARIABLES LIKE '%char%' results +----------------...
1
vote
2answers
66 views

Handling data encoding issues while loading data to SQL from Script (Notepad++)

I'm pretty sure this is not a SQL Server problem. I already asked a question HERE with an awesome explanation, BUT I still can't explain to the guys where I work that it has nothing to do with SQL ...
1
vote
3answers
160 views

PostgreSQL: difference between collations 'C' and 'C.UTF-8'

In PostgreSQL, what is the difference between collations C and C.UTF-8? Both show up in rows of pg_collation. Is it perhaps the case that C.UTF-8 is the same as C with encoding UTF-8 regardless or ...
0
votes
1answer
410 views

How can I search for a hex string in oracle?

There's a database record that is incorrect. The name field displays like this in a browser ( this is incorrect ): Thcˇodore And when I look at the record via SQL results, I see: Thc\xCB\x87odore ...
4
votes
2answers
370 views

Byte ordering for multibyte characters in SQL Server versus Oracle

I am currently in the process of migrating data from Oracle to SQL Server and I'm encountering an issue trying to validate the data post-migration. Environment Details: Oracle 12 - AL32UTF8 ...
1
vote
2answers
538 views

Bulk insert not retaining special chars of UTF-8 Encoded txt file Sql Server 2008

I have a stored procedure that bulk imports a text file and inserts it into my database. CREATE TABLE DBO.TEMP_STORE ( ID nvarchar(max), [MONTH] nvarchar(max), [...
4
votes
3answers
399 views

Unicode storage of \u202b RLE and \u202c PDE in a Unicode-aware database?

I'm building a new product for toponyms and in it the Arabic shows kinda like this: ^IArabic^I<202b>ﺰﻤﺑﺎﺑﻮﻳ<202c>^I<202b>ﺞﻫﻭﺮﻳﺓ ﺰﻤﺑﺎﺑﻮﻳ<202c>$ Actually not quite. This is a ...
1
vote
1answer
229 views

How to insert a Unicode character verbose into a varchar DB?

I need to insert this character '●' into a VARCHAR column of a MSSQL database with collation set as SQL_Latin1_General_CP1_CI_AS (or at least mock what my Python + Windows MSSQL Driver might have done)...
3
votes
2answers
1k views

SQL Server 2019 UTF-8 Support Benefits

I'm already quite comfortable with using COMPRESS() and DECOMPRESS() in an internal forum software for our company (Currently in SQL Server 2017), but trying to make the database as efficient as ...
0
votes
1answer
49 views

Users running ANSI scripts are having problems with special characers [closed]

Long story short. How can I fix users environments, to make them run our scripts using ANSI encode? The problem is, we send them scripts to run on their databases using ANSI encode. But some of ...
10
votes
1answer
3k views

Error starting SQL Server 2017 service. Error Code 3417

I have SQL Server 2017 installed on my computer. This is what SELECT @@VERSION returns: Microsoft SQL Server 2017 (RTM-GDR) (KB4293803) - 14.0.2002.14 (X64) Jul 21 2018 07:47:45 Copyright (C) ...
0
votes
1answer
278 views

How to configure DBeaver to save files without a BOM for UTF files?

I am looking for the equivalent setting as the PGADMIN setting Write BOM for UTF files which will cause saved files to be saved with/without the BOM. I've found settings in DBeaver when exporting ...
1
vote
2answers
444 views

Wrong sort order using utf8_unicode_ci and swedish characters like ö

I am confused about what encoding and collation I should use so my queries returns the correct sort order. Right now my encoding is UTF-8 Unicode and collation utf8_unicode_ci for both firstname and ...
1
vote
1answer
18 views

Table/Column Character Encoding and Indexes

Does table/column character encoding have any impact on how the MySQL optimizer chooses join indexes? I have a table that have the same columns and indexes on two servers, one server does a eq_ref ...
4
votes
1answer
4k views

Why do I get incorrect characters when decoding a Base64 string to NVARCHAR in SQL Server?

I've been looking at how to decode Base64 with SQL Server and after searching around many solutions online (some from here) seem to be based upon this sort of method. SELECT CAST(CAST('...
1
vote
1answer
46 views

Convert Character Encoding, and Check Encoding for Issues

I'm planning to convert a table from utf8 (_general_ci) to utf8mb4 (utf8mb4_unicode_ci). I want to make sure I don't have any malformed data after the conversion though so I was planning to duplicate ...
5
votes
1answer
1k views

Postgres - Encoding, Collation and CType

List of databases Name | Owner | Encoding | Collate | Ctype | -------------------------+----------+----------+---------------------------...
3
votes
2answers
561 views

Use non-printable characters in SQL Server BCP xml format file as TERMINATOR

I have a flat-text data file containing records which fields are separated by the non printable character "File Separator" (0x1c). I am trying to use SQL Server's bcp utility to load this data into my ...
2
votes
3answers
3k views

NLS_LANG and NLS_CHARACTERSET

I thought I read that NLS_CHARACTERSET has the encoding of the database and NLS_LANG the encoding of the client. Is that correct? Does that mean that the two encodings can be different? In some ...
0
votes
1answer
85 views

Change of NLS_LANG

Until not long ago, the NLS_CHARACTERSET and the NLS_LANG were both ISO88591. We deliver to production some scripts encoded in ISO88591. Since they changed their NLS_LANG in utf8, they have a problem ...
1
vote
1answer
125 views

Is there any string compaction option for PostgreSQL?

I have a table like this: CREATE TABLE BLOG( CATEGORY CHAR(32) ); Category is a column, which stored enum values. For example, I have 1 millions of rows and 10 unique values of Category. Let'...
2
votes
2answers
385 views

DB2 on Windows: Fix encoding in db2cmd and for JDBC

I'm running a DB2 v10.5 FP 7 on a Windows Server 2013 and there is a problem with the encoding setup: The windows is set to use German(Switzerland) as region setting and input language. chcp: Active ...
3
votes
2answers
3k views

Character Conversion Issue Importing From CSV File

After doing a load of CSV files, there are various words that are "written" to the database incorrectly. Some examples: Diã¡ria should be Diária Crã©dito should be Crédito Ligaã§ãµes should be ...
5
votes
2answers
2k views

SQL Server Bulk Insert properly interprets some Unicode characters but not others?

For some reason the MS SQL Server 2016 bulk insert misinterprets/translates Unicode characters: C9 (É) into 2B (+) A1 (¡) into ED (í) A0 ( ) into E1 (á) AE (®) into AB («) CB (Ë) into 2D (-) D1 (...
1
vote
0answers
71 views

Merge Replication Business Logic Handler Incorrect Encoding when connecting to SQL Server

We have a SQL Server Merge Replication configuration that uses a business logic handler to do conflict resolution on a datetimeoffset column. All we're doing is a copy of the Subscriber or Publisher ...
2
votes
0answers
877 views

Why is my UTF-8 document raising UTF-8 encoding errors in Azure Data Lake Analytics?

I have a document that was compressed in gunzip from a unknown source system. It was downloaded and decompressed using a 7zip console application. The document is a CSV file that appears to be encoded ...
1
vote
1answer
1k views

How to Create a PostgreSQL database with different Encoding?

I want to create a New Database that will store data in German(i.e Text with accents etc). Hence I tried to create it using the following SQL: CREATE DATABASE testdb WITH ENCODING='LATIN1' ...
1
vote
1answer
195 views

What is the equivalent of the SQL Server CHAR function?

SQL Server has CHAR ( integer_expression ), so I can do SELECT CHAR(65) What is the equivalent on Postgresql? I have tried SELECT E'\0x65'; but I get ERROR: invalid byte sequence for encoding "...
4
votes
1answer
567 views

Collate issues with wrong characters

Well, the problem is well known, but I'm looking for a smarter solution if there's one. For some reason the system doesn't recognize some characters and I can't compare the columns Here is an ...
16
votes
4answers
7k views

Why does the varchar datatype allow unicode values?

I have a table with a varchar column. It is allowing Trademark(™), copyright(©) and other Unicode characters as shown below. Create table VarcharUnicodeCheck ( col1 varchar(100) ) insert into ...
2
votes
2answers
3k views

Remove characters from PostgreSQL database causing encoding errors

I have a PostgreSQL 10 database that uses WIN1252 encoding. One of my columns has values that cause conversion errors when running a select from pgAdmin 4: SELECT myfield FROM mydb.myschema."MYTABLE"...
0
votes
1answer
255 views

UTF-8: showing correct in database, but not in webpage

When i upload data from php post forums "हिन्दी" language, changed into "हरिà" in database but looking correct in webpage, and when i upload data through phpmyAdmin from csv file "हिन्दी" ...
3
votes
1answer
4k views

How to be sure that a SQL Script is running with ANSI encoding?

Maybe this is a stupid question or I'm asking it in the wrong way. How can I be sure that a script ( with thousands of lines ) is running with ANSI encoding? Let's say we created a script using ...
1
vote
1answer
83 views

Invalid characters in xml [closed]

I have just downloaded a large xml file and most of characters appear as "�". I changed the encoding but there was no improvement. Anyone knows what this might be due to? One fishy thing: the file ...
10
votes
1answer
7k views

Encoding issue with SQL Server VARCHAR column retrieved in Python

We recently had an issue with encoding related to a field that's being stored as a varchar(120) in SQL Server. In SSMS, the varchar appears as: "Who Killed JonBen‚t?" However, when it's brought ...
5
votes
1answer
1k views

How do I list all available encoding types available in postgreSQL?

SELECT datname, pg_encoding_to_char(encoding) FROM pg_database; …lists all the databases, each with its encoding type. However, I am trying to find out what are all the encoding types available in ...
1
vote
1answer
1k views

Saving into a VARCHAR(MAX) throwing error in some scenarios

We are noticing an error like this: InterfaceError: ('HY104', 0, '[Microsoft][SQL Server Native Client 11.0] Invalid precision value', 10226) It happens when we insert a certain length of characters ...
4
votes
1answer
606 views

Two tables with two different charsets

I have one table which stores input from user, so to allow national characters I've set charset to utf8mb4. Then I have second table, which can grow really large (100+ millions of rows) and I'm 100% ...
4
votes
1answer
5k views

Using HASHBYTES() yields different results for nvarchar and a variable

I'm using server-side hashing to transmit passwords, then running PBKDF2 in the database to store the hashed password + salt combination. Hashing nvarchar(max) and a @variable holding the same value ...
0
votes
1answer
4k views

How to export data in Oracle database into CSV file with UTF-8 format?

I have an CSV file with WE8ISO8859P1 encoding generated from Oracle Stored Procedure. Is there any script line to define the output of CSV file in UTF-8 encoded format? CREATE OR REPLACE PROCEDURE ...
0
votes
1answer
653 views

Postgres upgrade and converting latin1 to utf8

I'm pretty new to PostgreSQL, but I've been tasked with doing an upgrade. The current system runs on CentOS 5.11 and is Postgres 8.1.23 with a default encoding of latin1. The new server is OpenSuSE ...
3
votes
1answer
221 views

Migrating from Postgres 8.4 to 9.4 in different data centers

I'm trying to migrate a Postgres 8.4 database in a Brazilian machine to Amazon RDS Postgres 9.4 in North Virginia. The database uses encoding LATIN1, tablespace pg_default, lc_collate and lc_ctype C. ...
3
votes
2answers
522 views

Database keeps using utfmb4 instead of utf8

Using MariaDB 10.1. The default encoding is utf8mb4, but I have an existing database that uses utf8. So in order to re-use the same DB, I would have to keep using utf8 for now. So I set the server ...
3
votes
1answer
201 views

ERROR: requested character too large for encoding

I'm using PostgreSQL 9.6 on Linux. I got an error when I do a test on chr() function. postgres=# select chr(1199111); ERROR: requested character too large for encoding: 1199111 postgres=# select ...
2
votes
1answer
3k views

Character Encoding in Postgres

I have UTF8 data stored in Postgres. I can query it with a Perl script and display it properly. When I use the Postgres client, it displays like the encoding is off. I'll see characters displayed ...
1
vote
1answer
428 views

sp_send_dbmail is populating 00 bits between all characters in a result set

I'm using sp_send_dbmail in a SQL agent job triggered nightly. The job queries our database and checks for product price updates, if there are some, it will send them in an email as an attachment to ...
3
votes
2answers
4k views

Not displaying special characters imported from a file

The scenario is a SQL Server instance, a database which is data fed using BULK INSERT operations mainly, and some of the texts inserted contain special characters like ñ because I am working in a ...
1
vote
0answers
326 views

Fixing a UTF8 encoded database that was dumped using an incorrect encoding?

The question is probably a little vague so let me elucidate: System OS is Ubuntu Linux 14.10 LTS The Servers default encoding is Latin1 The database default encoding is/was UTF8 The data was ...
4
votes
1answer
5k views

Equivalent of UTF8_UNICODE_CI collation in PostgreSQL

I would like a column in a table inside a PostgreSQL database (I am using version 9.6). I know of the UTF8_UNICODE_CI collation on MySQL, so I tried: CREATE TABLE thing ( id BIGINT PRIMARY KEY ...
0
votes
1answer
118 views

psqlODBC access encoding problem

My local machine: Windows 7, 32-bit. I have installed on it the psqlODBC for 32-bit. MS Access is 32-bit as well. I have tried both Unicode and Ansi connections to... The server: a 64-bit Ubuntu, and ...