Top tag
Next privilege 250 Rep.
View close votes
1 5
~7k people reached

  • 0 posts edited
  • 0 helpful flags
  • 6 votes cast
comment Getting new and updated data with an archive column
As I understood the requirement, you have a set of "all of the rows processed" (the Archive=1 candidates) from which you want to exclude "any rows that might have been inserted or updated after the initial select" (the Archive=0 and Archive=2 rows). So the Lookup(s) would be connected to the Cache(s) as loaded by the Cache Transformation(s) in the earlier Data Flow(s). The Lookup would not be connected to a database table.
comment Database/Storage engine suggestion for a project involving heavy inserts?
The best resources (books, articles, training) on Data Warehousing IMO are from the Kimball Group.
comment SSIS Design consideration with object variables
From my experience the minimal effort to set up SQL tables pays off even during your Unit Testing phase. You can easily write and reuse test SQL queries to check and compare your results at each iteration and for varying inputs. SQL has no problem handling multiple inputs and parallel executions - just dont lock the target tables. In that scenario I would be more worried about running out of RAM with your current design. SSRS is optional - once you have the data in tables you can use a wide range of tools to get it out.
comment Convert rows to columns in SSIS. Unpivot?
Elaborated my answer - see above.