1

I need to create an extraction job from an Informix database, the requirement is that I need to enable transactions in SSIS. All of the MSDTC settings of the target machine and the SQL Server database are working, and the RPC service is listening on port 135, both firewall are disabled and no antivirus is installed.

The Informix server is on a Linux Red Hat 6.5, where the SELinux is permissive, the IpTables disabled. I also ran the cdotnet.sql and coledbp.sql scripts as directed by an IBM article to configure linked servers in SQL Server - and it worked.

The connection between the development machine and Informix is through an ODBC connection, which is also successful in querying the tables.

But when the job is started the following error occurs in SSIS:

Error: 0xC001402C in 1-Load CTLANCTO: SSIS Runtime Failed to Enroll OLE DB Connection in a Distributed Transaction with Error 0x8004D01B "The transaction manager is not available."
Error: 0xC0202009 in 1-Load CTLANCT: SSIS Error Code DTS_E_OLEDBERROR. OLE DB error. Error code: 0x8004D01B.
Error: 0xC020801C in Load CTLANCTO On Stage, OLE DB Source [16]: SIS Error Code DTS_E_CANNOTACQUIRECONNECTIONFROMCONNECTIONMANAGER. The call of the AcquireConnection method for the "OLEDB informix" connection manager with error code 0xC0202009 failed. Error messages may have been posted earlier with more information about why the call failed for the AcquireConnection method.
Error: 0xC004701A in Load CTLANCTO On Stage, SSIS.Pipeline: OLE DB Source Fails in Pre-Run Phase with Error Code 0xC020801C.
Information: 0x4004300B in Load CTLANCTO On Stage, SSIS.Pipeline: "Stage" recorded 0 lines.
Information: 0x40043009 in Load CTLANCTO On Stage, SSIS.Pipeline: The cleaning phase is starting.
Task failed: Load CTLANCTO On Stage
Information: 0x4001100C in Load CTLANCTO On Stage: Overriding the current distributed transaction.
Information: 0x4001100C in 1-Load CTLANCT: Overriding the current distributed transaction.
SSIS Package "C: \ Projects \ ETL \ Daily LoadDRE \ Daily LoadDRE \ 1-Load CTLANCTO.dtsx" Finished: Failed.
The program '[796] DtsDebugHost.exe: DTS' was closed with the code 0 (0x0).

Although I have little knowledge on Linux, I tried to connect the development machine with the Informix server on port 135 via telnet, and the result is::

Connecting to x.x.x.x ... Could not open host connection in port
to 135: Connection failed

Then by running the rcpinfo command, it has the following:

program version netid     address                service    owner
    100000    4    tcp6      ::.0.111               portmapper superuser
    100000    3    tcp6      ::.0.111               portmapper superuser
    100000    4    udp6      ::.0.111               portmapper superuser
    100000    3    udp6      ::.0.111               portmapper superuser
    100000    4    tcp       0.0.0.0.0.111          portmapper superuser
    100000    3    tcp       0.0.0.0.0.111          portmapper superuser
    100000    2    tcp       0.0.0.0.0.111          portmapper superuser
    100000    4    udp       0.0.0.0.0.111          portmapper superuser
    100000    3    udp       0.0.0.0.0.111          portmapper superuser
    100000    2    udp       0.0.0.0.0.111          portmapper superuser
    100000    4    local     /var/run/rpcbind.sock  portmapper superuser
    100000    3    local     /var/run/rpcbind.sock  portmapper superuser
    100024    1    udp       0.0.0.0.173.232        status     29
    100024    1    tcp       0.0.0.0.132.42         status     29
    100024    1    udp6      ::.223.153             status     29
    100024    1    tcp6      ::.169.187             status     29

Does this mean that the RPC service in Informix is running on port 111? Is it necessary to change to 135?

The important detail is that when I disable transactions option in SSIS, extraction successfully occurs.

Test thousands of solutions over the internet and nothing worked, my last hope is here.

  • Dumb question: why do you want a transaction when you're doing an extract? All you're doing is reads, right? Surely you don't want to hold locks across all the source tables while you're doing an extract. – Brent Ozar Jan 15 '18 at 12:15
  • No problem, working with transactions during the extraction, we ensure that in case of failures the destination database will not have dirty information. This work occurs at dawn, so blockages can occur without impact. – Rafaells Jan 15 '18 at 12:20
  • Then the answer is using a staging database and a 2-step process. First, do all of the reads out of the source database, writing the data into a temporary staging database. Then, do your loads from staging into the target separately. If you want to use a transaction to wrap your loads into the target, that's a separate discussion. That also means if the load fails, you can retry from staging, and still not impact the source systems (or have to poll them again.) – Brent Ozar Jan 15 '18 at 12:54
4

From the comments, it sounds like you're trying to:

  • Load from Informix into SQL Server
  • Use a distributed transaction to wrap the entire load
  • Roll back and retry the loads if they fail

In that case, the answer isn't a distributed transaction - it's a staging database and a 2-step extract/transform/load (ETL) process:

  1. Select the data out of Informix into a temporary staging database (this could be in SQL Server - but don't bother with a transaction)
  2. Move the data from your staging database into your final destination - and if you do want to use a transaction to accomplish this, that's a separate debate, but at least it'll work

Benefits of this approach include:

  • If the Informix read portion fails for some reason and you have to retry it, the production data warehouse users still aren't affected
  • If the production data warehouse (SQL Server) write portion fails for some reason and you have to retry it, the Informix users aren't affected either

I know that you're going to say, "But I need to write it this way." Thing is, the reason you're not finding any results for your question out there is that people don't write it that way. As your data grows, your server grows, and your career grows, you're going to need to extract data from multiple sources and you won't be able to take locks across all of them at once to accomplish the load. That's why the ETL (extract/transform/load) design is so popular.

  • We have a stage area, it is at this stage that this error happens. Our idea is to have the stage table synchronized with the source table in Informix. So having the option of transactions, will keep the stage area always safe with reliable data in case of errors during loading. – Rafaells Jan 15 '18 at 16:48
  • 1
    That idea (keeping your staging area in sync with your source using transactions) isn't feasible or scalable long term. Either gather all of the data, or build change detection (like give me the rows in the Sales table since 10:15PM, and the rows in the SalesPerson table since 10:11PM, that kind of thing.) – Brent Ozar Jan 16 '18 at 10:55
  • So you think it's more logical to have a log table to keep track of the latest loads and truncate the stage tables before the new loads? So have transactions only between the stage and data warehouse areas? – Rafaells Jan 16 '18 at 14:38
  • 1
    Yep, exactly. But don't just take my advice here - it's good time to pick up a book on data warehousing. It's a pretty big subject matter area, and if you're just hearing that now, I'm guessing you haven't been to a training class, watched webcasts, read books, etc on how to do fast data warehouse loads. Consider Andy Leonard's books on SSIS, for example. – Brent Ozar Jan 16 '18 at 15:36

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.