Tag Info

New answers tagged

0

The IRI Workbench (Eclipse GUI) supports bulk (multi-table) extraction, mapping (transformations, if needed, with rules), and bcp loads; specifically, its' reorg wizard is mentioned here. I wouldn't start with 200 sources at once, but see how it goes for the first 10 to determine if its multi-script approach (which I like for portability and easy ...


0

Depending on the volume, if you use an affordable high-speed tool for data integration like IRI CoSort (which can use either bulk-extracted/loaded flat-file data or ODBC connections), you'll be near real-time anyway (with its speed, and without in-DB transforms). To be closer to real time per the article, you can 'regulate flow' via CDC in a data- (rather ...


0

It's also a matter of money. Where data volumes are high as you point out, flat-file based solutions like Ab Initio and DataStage Parallel Extender are indeed faster, but can be mid-to-high six-figure propositions. IRI CoSort is very ETL-centric (per their ELT comparison), and the only affordable way I've seen to address the transformation volume with ...


0

NB:- I am giving the sql syntax of MS SQL Server(i am only familiar with).I guess you can develop from it. First, you have to select the data using sql. Usually UNION returns only distinct raw SELECT LastName, FirstName, Address, City, State, Zip FROM HospitalOne UNION SELECT LastName, FirstName, Address, City, State, Zip FROM HospitalTwo ORDER BY ...


0

You would need to something called data cleansing. There are tools within which such logic can be written for example IDQ or IBM QualityStage if not then you would need to write Stored Procedures while doing the ETL to de-duplicate/clean up the data and then load the final list into DWH.


1

Pentaho have an open-source tool called Kettle that we use for ETL development with Postgres. It is quite good and very comprehensive. http://community.pentaho.com/projects/data-integration/


1

First of all: Why the heck would you move data FROM SQL Server TO Postgres in order to use SQL to manipulate it? Do you realise that Postgres does not have a parallel query executor while SQL Server has? This means that even on a tiny server, a data warehouse data manipulation query will run several times faster on SQL Server than it does on Postgres . There ...



Top 50 recent answers are included