DB2 selects break after migrating from RHEL5, V9.7 to RHEL6, V10.5

Question

I'm migrating a bunch of perl CGI scripts from an older RHEL5 installation to a RHEL6 server. On the old server, DB2 V9.7 was installed, the new server has V10.5. No databases on the servers themselves, just several connections to remote databases.

I can access one of these remote databases, which is running on AIX, without problems, so the installation itself should be ok.

I can connect to another of the remote databases, without problems on the old RHEL5 server. On the new server, i can connect ok, but as soon as i try to select something, the connection breaks with the following error:

[IBM][CLI Driver] SQL30081N  A communication error has been detected. 
Communication protocol being used: "TCP/IP".  
Communication API being used: "SOCKETS".  
Location where the error was detected: "10.199.252.155".
Communication function detecting the error: "recv".
Protocol specific error code(s): "*", "*", "0".  SQLSTATE=08001

Note that the IP address, 10.199.252.155, is the IP of the database server, not mine, so it seems that the DB2 server detects some error and reports that to me, instead of the client detecting the error itself.

On both of my servers, the commands to catalog the remote database (issued 2 years ago on the RHEL5 server, issued a few days ago on RHEL6) were

CATALOG TCPIP NODE TCP0000 REMOTE iaixdb2i SERVER 3910 REMOTE_INSTANCE tdbi30 SYSTEM IAIXDB2I OSTYPE AIX;
CATALOG DATABASE DB2I21 AS DB2I21 AT NODE TCP0000;

The output of LIST NODE DIRECTORY, on both servers, shows the same entry:

Node name                      = TCP0000
Comment                        =
Directory entry type           = LOCAL
Protocol                       = TCPIP
Hostname                       = iaixdb2i
Service name                   = 3910

and LIST DATABASE DIRECTORY is almost the same - the old server has:

Database alias                       = DB2I21
Database name                        = DB2I21
Node name                            = TCP0000
Database release level               = d.00
Comment                              =
Directory entry type                 = Remote
Catalog database partition number    = -1
Alternate server hostname            =
Alternate server port number         =

while the new server has

Database release level               = 10.00

(the rest is identical).

When googling for my error codes, i found a lot of sites where a) the connection didn't succeed at all, or b) the connection broke for timeout reasons. Both these errors don't seem to apply to me. Also, those entries had some relevant error codes in the protocol specific section, not just "", "", "0" like i have.

So the only difference i see right now is the different database release levels. Which means the only problem i can think of is, that the database is still running under something older, and has issues with the V10.5 protocol. Does this mean i have to downgrade my client to 9.7, or is there a way to tell the newer client to use an older protocol level, by changing the database release level in the catalog? If so, what would be the commands to do that? Or is there anything else i should try?

Edit: Added info as requested by mustaccio. I wasn't prompted when i didn't pass user and password on the command line, so i had to modify the connect command to include them. Also, i changed username and password in this post - the real username starts with a # as well though.

Old machine:

$ db2 connect to DB2I21 user '#ABCDEF' using XXXXXXX

   Database Connection Information

 Database server        = DB2 z/OS 10.1.5
 SQL authorization ID   = #ABCDEF
 Local database alias   = DB2I21

$ db2 "select * from sysibm.sysdummy1"

IBMREQD
-------
Y

  1 record(s) selected.

$

New machine:

$ db2 connect to DB2I21 user '#ABCDEF' using XXXXXXXX

   Database Connection Information

 Database server        = DB2 z/OS 10.1.5
 SQL authorization ID   = #ABCDEF
 Local database alias   = DB2I21

$ db2 "select * from sysibm.sysdummy1"
SQL0805N  Package "DB2I21.NULLID.SQLC2K26.4141414141664164" was not found.
SQLSTATE=51002

$

Edit 2: even more information

Binding db2clipk.bnd works, but doesn't change the error message when i select from sysibm.sysdummy1.

Binding db2clpcs.bnd, as suggested by IBM for SQLC2K26 results in missing access rights; since this is a production database, i can't get the DBAs to change anything.

Ok, so i tried the test database, which has an identical setup, except the I is changed to J:

Node name                      = TCP0001
Comment                        =
Directory entry type           = LOCAL
Protocol                       = TCPIP
Hostname                       = iaixdb2j
Service name                   = 3910

Database alias                       = DB2J21
Database name                        = DB2J21
Node name                            = TCP0001
Database release level               = 10.00
Comment                              =
Directory entry type                 = Remote
Catalog database partition number    = -1
Alternate server hostname            =
Alternate server port number         =

has the same connection message

$ db2 connect to DB2J21 user '#ABCDEF' using XXXXXXXX

Database Connection Information

Database server        = DB2 z/OS 10.1.5
SQL authorization ID   = #ABCDEF
Local database alias   = DB2J21

and, which really amazes me, does not throw any error when i select, from the new machine!

$ db2 "select * from sysibm.sysdummy1"

IBMREQD
-------
Y

  1 record(s) selected.

Could this be a restriction on my IP address? If so, why does it allow me to bind? Or, is there a way for me to find out the patch levels of both databases? (It's going to be much easier for me to have the DBAs install a patch if i have a patch number, and a description that clearly states that this is what prevents me from selecting).

Edit 3: How it ended

Just in case anyone is interested or has a similar problem:

I got the DBAs to bind the V10.5 client packages. After that,

$ db2 "select * from sysibm.sysdummy1"

worked well against the productive database (DB2I21). My perl program still crashed.

Deinstalled DB2 Express C 10.5, installed the 10.5FP3 DS driver. Still the same problem, both databases work from the command line, the test database works with perl, the prod database doesn't.

I did a wireshark trace on the perl program against test and prod DBs - up to a certain point, they seem to be exactly identical, except the password exchange. After the client has executed the query and starts retrieving result values, the server just closes the TCP connection, without returning anything, not even an empty or error TCP packet.

I gave up at that point and installed the 10.1FP3 DS driver. Everything seems to work well with that one.

Answer 1

To avoid the SQL0805N error, try binding the CLI packages (you need to be connected): db2 bind db2clipk.bnd blocking all sqlerror continue grant public. The bind file is in the bnd directory under your DB2 client installation. If you don't have enough permissions to bind packages your DBA will have to do it for you.

The database server version shows in the connect command feedback: DB2 z/OS 10.1.5. There is no patch to be applied; you need to bind appropriate packages in the database for clients to work. This is a standard procedure when upgrading clients, because each new version of client software comes with its own packages that need to be bound to the database, so your DBA should not have any problem with that. Alternatively, he (or she) will tell you what client version you must use (for which the packages would already be bound).