How do I identify the column(s) responsible for "String or binary data would be truncated."

Question

I am generating some queries automagically with code I wrote to SELECT from a remote Pg database, and insert into a local SQL Server database. However, one of them is generating this error:

[Microsoft][ODBC SQL Server Driver][SQL Server]String or binary data would be truncated. (SQL-22001) [state was 22001 now 01000]

[Microsoft][ODBC SQL Server Driver][SQL Server]The statement has been terminated. (SQL-01000) at .\insert.pl line 106.

How do I find out what column is generating that error and lacks the length for the input? Is there a way to do this without brute force-guessing all the varchar?

Answer 1

No, it is not logged anywhere. Go vote and state your business case; this is one on the long list of things that should be fixed in SQL Server.

This was requested years ago on Connect (probably first in the SQL Server 2000 or 2005 timeframe), then again in the new feedback system, and now it has been delivered in the following versions:

• SQL Server 2019
• SQL Server 2017 CU12
• SQL Server 2016 SP2 CU6

In the very first public CTP of SQL Server 2019, it only surfaces under trace flag 460. This sounds kind of secret, but it was published in this Microsoft whitepaper. This will be the default behavior (no trace flag required) going forward, though you will be able to control this via a new database scoped configuration VERBOSE_TRUNCATION_WARNINGS.

Here is an example:

USE tempdb;
GO
CREATE TABLE dbo.x(a char(1));

INSERT dbo.x(a) VALUES('foo');
GO

Result in all supported versions prior to SQL Server 2019:

Msg 8152, Level 16, State 30, Line 5
String or binary data would be truncated.
The statement has been terminated.

Now, on SQL Server 2019 CTPs, with the trace flag enabled:

DBCC TRACEON(460);
GO

INSERT dbo.x(a) VALUES('foo');
GO
DROP TABLE dbo.x;
DBCC TRACEOFF(460);

Result shows the table, the column, and the (truncated, not full) value:

Msg 2628, Level 16, State 1, Line 11
String or binary data would be truncated in table 'tempdb.dbo.x', column 'a'. Truncated value: 'f'.
The statement has been terminated.

Until you can move to a supported version/CU, or move to Azure SQL Database, you can change your "automagic" code to actually pull the max_length from sys.columns, along with the name which you must be getting there anyway, and then applying LEFT(column, max_length) or whatever PG's equivalent is. Or, since that just means you'll silently lose data, go figure out what columns are mismatched and fix the destination columns so they fit all of the data from the source. Given metadata access to both systems, and the fact that you're already writing a query that must automagically match source -> destination columns (otherwise this error would hardly be your biggest problem), you shouldn't have to do any brute-force guessing at all.

Answer 2

Finally Microsoft has decided to provide meaningful information for String or binary would be truncated starting from SQL Server 2016 SP2 CU, SQL Server 2017 CU12 and in SQL Server 2019.

The information now includes both offending table column (fully qualified name) and the offending value (truncated at 120 characters):

Msg 2628, Level 16, State 1, Line x String or binary data would be truncated in table 'TheDb.TheSchema.TheTable', column 'TheColumn'. Truncated value: '...'. The statement has been terminated.

Answer 3

If you have access to run SQL Server Import and Export Wizard from SQL Server Management Studio (right-click database > Tasks > Import Data...), create a task that imports from SQL Client using your query as the data source to the destination table.

Before you run the import, you can review the data mapping and it will tell you which columns have inconsistent field types. And if you run the import task it will tell you which column(s) failed to import.

Sample Validation Warning:

Warning 0x802092a7: Data Flow Task 1: Truncation may occur due to inserting data from data flow column "NARRATIVE" with a length of 316 to database column "NARRATIVE" with a length of 60. (SQL Server Import and Export Wizard)

Answer 4

Ultimately, I could not find a way to get the column information without writing it myself.

This error message was generated by DBD::ODBC, you can also however use sys.columns (max_length) (I just don't know how).

I used code like this over my column list to get a list of arrays with two elements, the COLUMN_NAME, and MAX_LENGTH (documented in DBI column_info()).

my @max_lengths = map [ @{$_->fetchall_arrayref->[0]}[3,6] ]
    , map $dbh_mssql->column_info('database', 'dbo', $dest_table, $_)
    , @col_mssql
;

Then I caught the exceptions on INSERT and printed out something useful. In this example @$row is the data sent to sth->execute()

if ($@) {
        warn "$@\n";
        for ( my $idx=0; $idx <= $#{ $row }; $idx++ ) {
                Dumper {
                        maxlength => $max_lengths[$idx]->[1]
                        , name    => $max_lengths[$idx]->[0]
                        , length  => length( $row->[$idx] )
                        , content => $row->[$idx]
                };
        }
        die;
}

Also, please vote and upvote the other answer