Take the 2-minute tour ×
Database Administrators Stack Exchange is a question and answer site for database professionals who wish to improve their database skills and learn from others in the community. It's 100% free, no registration required.

Trying to build some sort of ETL process that copies rows from SQL Server into Elasticsearch indexing server.

My ETL process should be able:

  • Detect new rows inserted to the SQL Server table.
  • Detect updated rows in SQL Server table.

No so much interested in the ETL tools to use but rather Db patterns/script that will allow me to somehow query the SQL server to provide the above information.

share|improve this question
    
Change tracking, change data capture... don't re-invent the wheel. –  Aaron Bertrand Jul 7 at 18:30
    
Ok cool! Is that a specific feature of SQl Server 2008? A link would be appreciated so I know I'm looking at the right thing. –  user547 Jul 7 at 18:40
    
    
I agree with @AaronBertrand - here is an overview of the differences between the two main ways in SQL Server: technet.microsoft.com/en-us/library/cc280519(v=sql.105).aspx –  Sean Gallardy Jul 7 at 19:15
    
And this works on replicated table also? So I can enable Change Data Capture tracking on the table that receiving replication. We have a policy that all batch or ETL processes rum from an "archive" server and not directly from the production live. –  user547 Jul 7 at 19:47
add comment

Your Answer

 
discard

By posting your answer, you agree to the privacy policy and terms of service.

Browse other questions tagged or ask your own question.