Questions tagged [data-flow]

The tag has no usage guidance.

Filter by
Sorted by
Tagged with
0 votes
0 answers

Sink for user activity data stream to build Online ML model

I am writing a consumer that consumes (user activity data, (activityid, userid, timestamp, cta, duration) from Google Pub/Sub and I want to create a sink for this such that I can train my ML model in ...
0 votes
1 answer

How do you detect a change in grouping when using Partition By in SQL Server 2012+ and execute logic based upon the change in the same SQL statement?

If I have a SQL statement that contains something like, ROW_NUMBER() OVER(PARTITION BY school_area ORDER BY school_area) I get: 1 someschool1 schoolarea1 2 someschool2 schoolarea1 3 someschool3 ...
0 votes
1 answer

Power BI refresh dataflow keep a snapshot of data

my requirement is to be able to save the past state of an entity at a certain point of time. Currently when the dataflow is refreshed, the past state is lost and only the current one is displayed. Is ...
  • 1
3 votes
1 answer

SSIS Merge Join: Delete does not work

I've set up a Data Flow Task to sync a flat file with its destination table using a Merge Join with a Left Outer Join as described here (and elsewhere). Merge Join: Inserting and Updating works fine,...
  • 283
0 votes
1 answer

SSIS - Disable Task Based on Variable Value

I have an SSIS package that runs a number of Data Flow Tasks all at the same time. Each Data Flow Task is placed inside a Sequence Container. Before the Data Flow Tasks are run, a Script Task is ...
  • 503
2 votes
1 answer

How is SSDT Data Flow batching handled?

How does SSDT handle batching of Data Flow tasks? I am parsing a CSV that is about 1GB into SQL Server using SSDT. In the Data Flow there is a Flat File source, that goes to an OLE DB destination (a ...
  • 2,222