Database Administrators Stack Exchange is a question and answer site for database professionals who wish to improve their database skills and learn from others in the community. It's 100% free, no registration required.

Sign up
Here's how it works:
  1. Anybody can ask a question
  2. Anybody can answer
  3. The best answers are voted up and rise to the top

Trying to build some sort of ETL process that copies rows from SQL Server into Elasticsearch indexing server.

My ETL process should be able:

  • Detect new rows inserted to the SQL Server table.
  • Detect updated rows in SQL Server table.

No so much interested in the ETL tools to use but rather Db patterns/script that will allow me to somehow query the SQL server to provide the above information.

share|improve this question
    
Change tracking, change data capture... don't re-invent the wheel. – Aaron Bertrand Jul 7 '14 at 18:30
    
Ok cool! Is that a specific feature of SQl Server 2008? A link would be appreciated so I know I'm looking at the right thing. – user547 Jul 7 '14 at 18:40
    
    
I agree with @AaronBertrand - here is an overview of the differences between the two main ways in SQL Server: technet.microsoft.com/en-us/library/cc280519(v=sql.105).aspx – Sean Gallardy Jul 7 '14 at 19:15
    
And this works on replicated table also? So I can enable Change Data Capture tracking on the table that receiving replication. We have a policy that all batch or ETL processes rum from an "archive" server and not directly from the production live. – user547 Jul 7 '14 at 19:47

Your Answer

 
discard

By posting your answer, you agree to the privacy policy and terms of service.

Browse other questions tagged or ask your own question.