1

I am using SQL Server 2012's Import wizard to import data from DatabaseA to DatabaseB.

  • Source: DatabaseA table A
  • Destination: DatabaseB table B

I have written a source query that does the job fine, but I have only 2 options

  1. Delete destination data - and pull in everything
  2. Append data - but this creates duplicates (i.e. pulls in everything all over again)

So I wanted to try modify my source query to be something like below

select col1, col2, col3 from A where col1 not in (select col1 from 
DatabaseB.dbo.B)

In other words I want to test that the data I am pulling from DatabaseA's table A does not already exist Database B's table B where col1 is a unique value.

I would like to be able to do this over and over again, only ever pulling across the delta data

Whenever I try this I get an error stating

Deferred prepare could not be completed.
Statement(s) could not be prepared.
Incorrect syntax near the keyword 'in'. (Microsoft SQL Server Native Client 11.0)

Some extra things worth noting (might give some clues)

  1. DatabaseA and DatabaseB are on separate servers
  2. DatabaseA.A.col1 is an identity column
  3. DatabaseB.B is a table I created to match the columns of A, it has no identity column, or primary keys etc as it is only a staging table for a migration script I will run after
1
  • No, it is not a linked server
    – Crudler
    May 21, 2018 at 14:33

1 Answer 1

1

The Import/Export wizard builds out an SSIS package behind the scenes to perform EL (Extract and Load). It really doesn't have a Transform capability since it's a streamlined tool.

What you're trying to do is conditionally load data based on existence. Which is fine, that's a common pattern.

You're attempting to solve it by a non-working query.

select col1, col2, col3 from A where col1 not in (select col1 from 

DatabaseB.dbo.B)

This assumes that DatabaseB is on the same instance as A. If you established a Linked Server between A and B, then your query would work - whether it performs at acceptable levels is a different matter.

select col1, col2, col3 from A where col1 not in (select col1 from 

Server2.DatabaseB.dbo.B)

A more SSIS native way of doing this would be to add a Lookup between the Source and Destination that matches based on col1 but that cannot be done using the Import Export wizard. You'd have to edit the package using Visual Studio/SQL Server Data Tools (SSDT).

If you absolutely cannot edit the package for reasons, then you have to build 2 packages. The first will export all the Col1 IDs from Destination back to Source. You can then modify the source query to reference the local cache of ids instead of trying to cross servers.

Good all purpose read -> Stairway to Integration Services I think step 3 is an incremental load which is what you're looking for

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge that you have read and understand our privacy policy and code of conduct.

Not the answer you're looking for? Browse other questions tagged or ask your own question.