4

I have a data flow in a 2012 SSIS package where I'm trying to use a temp table as the data source. I'm using an OLE DB data source with a SQL Command Data access mode.

My code looks like this:

CREATE TABLE #Checksums (DBName sysname, CheckSum bigint)
-- Code that loads the Checksum table here
SELECT DBName, CheckSum FROM #Checksums

In order to get the data source to pull the list of columns I temporarily put the following:

SELECT CAST(NULL AS sysname) DBName, CAST(NULL as bigint) CheckSum

That got the columns created and I was able to map to my destination.

I have ValidateExternalMetadata set to false on the data source and DelayValidation set to true on the data flow.

My code works correctly in SSMS however every time I run the package it errors out with the following:

[OLE DB Source [39]] Error: SSIS Error Code DTS_E_OLEDBERROR.  An OLE DB error has occurred. Error code: 0x80004005.
An OLE DB record is available.  Source: "Microsoft SQL Server Native Client 11.0"  Hresult: 0x80004005  Description: "Invalid object name '#qtemp'.".
[OLE DB Source [39]] Error: Unable to retrieve column information from the data source. Make sure your target table in the database is available.
[SSIS.Pipeline] Error: OLE DB Source failed the pre-execute phase and returned error code 0xC020204A.

Is it possible to use a temp table like this in a data source? If so is there a trick I'm missing?

5
  • 2
    Are you setting the RetainSameConnection on the Connection Manager to TRUE? See if this helps: stackoverflow.com/questions/5631010/…
    – RLF
    Aug 4, 2014 at 18:42
  • 1
    @RLF I'm doing this in a loop hitting multiple servers will that affect this? Aug 4, 2014 at 19:09
  • 1
    @KennethFisher Sorry, I have never tried to hit multiple servers in a loop from SSIS. It seems logically possible to have an outer SSIS package call a sub SSIS-package that would connect to and retain a connection for a single server, then close that connection. But it is over my SSIS head.
    – RLF
    Aug 4, 2014 at 19:32
  • 1
    @RLF I've done the inner/outer package thing but prefer to use expressions to change the server name of the connection manager in a for each loop. I'll give the other a shot. Maybe since it's change server names that will force a connection change at the appropriate time. Aug 4, 2014 at 20:17
  • 2
    @KennethFisher Jaime Thomson (SSIS Junkie) shows Using temporary tables as well as on SE - How to create a temporary table in SSIS control flow task and then use it in data flow task?
    – Kin Shah
    Aug 5, 2014 at 3:18

4 Answers 4

3

These steps helped me:

  1. Write the final result set into a table.
  2. Script that table as CREATE into a new New Query Editor Window.
  3. Remove everything except the open and close brackets that define the columns.
  4. Wrap that into another pair of brackets.
  5. Recompose the calling of your SP from

    exec p_MySPWithTempTables ?, ?
    

into

exec p_MySPWithTempTables ?, ? with result sets
(
    (
        ColumnA int,
        ColumnB varchar(10),
        ColumnC datetime
    )
)
1

The temp table will confuse SSIS. To get around the issue, write your code like this:

Use Database
Begin
    Exec('Create Table ##Table(Field Integer)') With Result Sets None;
    Exec('Select Field From ##Table') With Result Sets
    ((
        Field Integer
    ));
End

This allows the use of the temp table and provides the metadata to SSIS.

5
  • 1
    Dropping local temporary tables is handled automatically when they go out of scope, but you are suggesting a global temporary table, and the behaviour is different for those. So perhaps this will resolve the OP's issue, but won't it introduce another one?
    – Andriy M
    Dec 9, 2015 at 6:47
  • 2
    Please explain "confuse". What is confusing and why is it "confusing"? Dec 9, 2015 at 8:35
  • @AndriyM The global temp table will drop automatically when the last session which has an open transaction on that table will disconnect. In the SSIS scenario that most often means after the package is executed. So Chris's solution is fine.
    – ajeh
    Jul 5, 2017 at 13:51
  • @ypercubeᵀᴹ It might confuse SSIS since it is saved on a server in tempdb database, see my answer here. If you now loop over many servers, as the questioner tells us, you need to call the connection of the right server again before you write to the temporary table. But that is not how temporary tables are meant to be, they should rather work like CTE:s, staying in the same server, and you would not need to care about the connection when you save to a temporary table. I am just guessing this, it might be one thing out of many. Mar 28 at 1:12
  • @ajeh This is wrong. A temp table is dropped as soon as you step out of just one DFT item to the next. Even if you do not close the connection, the table is always gone after the first item. I tested this with a Script Component, but it should be the same with a Data Source. Apr 7 at 15:34
0

I know this is an old post, but I thought I would offer another solution that I have found helpful not just in SSIS, but in Management Studio Import Wizard, PowerPivot, etc.. If you don't need to index your temp table, you could always use table variables instead. As long as you use the "set nocount on" hint at the beginning, you should be able to use this in your SSIS data flow source. Your new code should look something like this...

set nocount on

declare @Checksums table (DBName sysname, CheckSum bigint)
--Code that loads the Checksum table here
select DBName, CheckSum from @Checksums
2
-1

The answer to the question is "No, it cannot be done"

A temp table is dropped as soon as you step out of just one DFT item to the next. Leaving the Data Source item where you create a temp table will already drop it again. Yet, if you work with a Script Component, you can work around this and keep the temp table in memory inside the connection manager by not closing but releasing it at the end of "Script Component 1" so that the temp table is available in "Script Component 2" once you acquire the released connection again, see DFT drops connection and its temporary table after leaving any DFT item, tested with two Script Components. How do I keep the temp table alive?.

I also found remarks that assure this viewpoint, see Using temporary tables in SSIS flow fails:

The asker's remark:

I am using temporary tables because i dont want to use OleDbCommand objects to perform updates on a row by row basis. I drop new rows into a temp table and then issue a merge to update the staging table - i am happy that they disappear after the job is finished

The remark on this by Panagiotis Kanavos:

They disappear when the connection closes, not when the job is finished. The connection will close when you move from one dataflow to the next, or when you move from the SQL script that created the table to then next script or flow. Which is why the package is broken - there is no table there to bind to or validate.

To be sure about this, I tested this with two Script Components after each other, and when I released the connection and acquired it again, the tmpTable came back into the "tempDB" database, fully filled. With a Data Source as a source, you do no longer have the choice to release the connection, and instead, it gets closed, as we read from the question itself. If it released the connection instead, this question would not pop up at all, and there is no way to switch from closing to releasing the connection in the settings of the Data Source item.

Workaround 1: Make and fill the temp table in a Script Component and pass its data over the Script Component output

There is a way to make a temporary table in a Script Component and loop over the Row object to pass it as the output to the next DFT item. This is not an answer to the question since you cannot do this with the mere Data Source item. And even if you take the Script Component instead, the temp table itself will not survive in the tempDB database or at least seems to get hidden in between the Script Components. But it is kept in memory inside the connection itself and pops up again once you acquire the connection in the next Script Component. For the full code, see again the link from above: DFT drops connection and its temporary table after leaving any DFT item, tested with two Script Components. How do I keep the temp table alive?.

To see how you can create and fill the temp table, see: How do I create and fill a temporary table with incoming data from a Data Source item without leaving the SSIS Data Flow Task C# Script Component?.

Workaround 2: Make a temp table in the Control Flow, then you can take it up in one Data Flow Task item before it gets dropped (or you redo the workaround 1 there and keep it alive with a Script Component by releasing the connection without closing it)

If you begin with making a temp table in a Control Flow "Execute SQL Task", you can reach this temp table in the DFT.

The bad news: You can reach it only one time for just one DFT item. If you need it again and again inside the same DFT, see "Workaround 1".

The good news: There is no need to deal with variables, stored procedures or other workarounds. Take the "tempdb" system database as the ADO.NET connection on project level and set the "RetainSameConnection" connection property to True. All of your temporary tables are always just saved in that "tempdb" system database.

See a guide on this with many screenshots at How to create a temporary table in SSIS control flow task and then use it in data flow task? - Stack Overflow.

SSIS can reach temporary tables on the "tempdb" database and deal with these tables as if they were not temporary. The temporary tables stay alive within the Control Flow until the run stops or until you step from one Data Flow Task (DFT) item to the next unless you follow workaround 1, or if you stop the debugging, see How to see temp table created by code in sql server?. You can choose them as a normal destination or source in the editor while you are still planning if you create them during your SSIS work.

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.