'Extract, Transform and Load': Data load processing, normally in the context of data warehouse systems.

learn more… | top users | synonyms

0
votes
2answers
25 views

Can data be replicated from a slave MySQL back to a master without the slave containing all the master data?

We have a PHP/Symfony 2 application using MySql as the database. The database contains a large amount of data. The application is deployed to a separate server(s) and an empty/clean version of the ...
1
vote
1answer
17 views

Setting the column width of a column in a table created by logparser

I'm trying to use LogParser for ETL. I am trying to import this file with the following command: "c:\Program Files (x86)\Log Parser 2.2\LogParser.exe" "SELECT Field1 AS CountryCode, Field2 AS Zip ...
1
vote
1answer
27 views

Are there best practices or standardized syntax for writing transformation logic?

This may not be the appropriate forum, but its the closest I could find. I write a lot of source to target mappings for work. These documents just describe the operations/logic/transformations/etc. ...
1
vote
1answer
137 views

Dimensional Modeling and ETL in Redshift

I have been researching Amazon's Redshift database as a possible future replacement for our data warehouse. My experience has always been in using dimensional modeling and Ralph Kimball's methods, so ...
3
votes
0answers
727 views

SSIS Recordset Destination error

I have an SSIS package targeted at SQL Server 2012. I have it deployed into the Stored Packages in Integration Services and then have a SQL Server Agent job which executes it. The first part of the ...
2
votes
0answers
704 views

How to do automated archiving for a SQL Server database based on interpretation of database metadata?

I am looking for a solution giving me automated archiving of SQL Server databases based on a smart interpretation of table relationships that is able to archive records from related tables minimizing ...
1
vote
0answers
49 views

Connection refused when inserting

Description: Machine 1: Linux, PostgreSQL 9.3, on local, 10.12.10.123 Machine 2: Linux, PostgreSQL 9.3, on cloud, 104.10.124.123 Configuration (on both): + postgresql.conf : listen_addresses ="*" + ...
1
vote
0answers
133 views

ETL - Moving data from Postgres/Oracle to a TripleStore (RDF)

I'm looking for pointers on how to handle an ETL process that I have to implement for my organization. I know how to query SQL database but i'm not SQL DB specialist. I work more in the sphere of the ...
1
vote
0answers
158 views

Transfer data from DynamoDB to SQL server 2008?

I want to transfer data from amazon dynamo DB to SQL server 2008. What are the tools or methods which can be used to do this ?
1
vote
0answers
209 views

ETL import more dimensions from only 1 column

I was requested at work to learn ETL. I have already created a database without using ETL with some dimensions and consolidations, etc, containing above 1 billion numerical values. Now I have to ...
0
votes
0answers
30 views

Tools to insert data faster in MySql database

I am trying to insert data from a flat file which is around 9 GB with about 15 million rows, I used SQl-server 2012 for that. I noticed data insert is much faster with sql server and it took around 30 ...
0
votes
0answers
15 views

Does an Object literal always mean the Predicate and Object form a key-value property on the Subject?

A triple is Subject-Predicate-Object, e.g. Post Office - Not Open - Sunday. Consider another example, Post Office - Size in Square Feet - "4000" For this example, I have two observations: the ...
0
votes
0answers
32 views

ETL flow in Fact tables

I've got a simple theoretical question... If we discuss the principles, elements, design and implementation of a typical ETL flow for a dimension table then, in my understanding, (in short) I'd say ...
0
votes
0answers
73 views

Best way to run an ETL on a production MongoDB database

We have the following environment: 3 servers + 3 replica set servers Each server has 3 shards Our main collection has around 40,000,000 documents that average at around 6kb Our shard key is ...
0
votes
0answers
49 views

Tracking row level status of SSIS ETL process

I have an ETL Process in SSIS in which I am loading around 150 tables in each run - Truncate and Insert. I have four packages each from different sources. Each package loads different tables. ...
0
votes
0answers
36 views

Tables with same schema in DB for different sources. Bad pattern?

The main purpose of the system being developed is to collect data from different sources, process and standardize it, and then add it to the DB. After information from all sources is collected it ...
0
votes
0answers
138 views

Selectively load data using Oracle 11g r2 External Table Preprocessor

I am using the Preprocessor feature in Oracle 11g r2 to load several data files in a table. However, the raw files seem to have certain rows that shouldn't be imported with the rest. For example: ...
0
votes
0answers
816 views

ssis ado.net destination has failed to acquire the connection

I am Transferring data from sql server source db to sql server destination db (cross db). but some times getting error "ssis ado.net destination has failed to acquire the connection.The connection may ...
0
votes
0answers
45 views

Merge data using Integration Service

Please Consider this scenario: I have a table in my database. I want move this data in my OLAP database using SSIS.I can move all record from my table to OLAP database.The problem is I don't know how ...
0
votes
0answers
71 views

Inserting Records - How to get next incremented PK for each insert

I'm using spoon to look for Devices that have several attributes that define its location, and then create a Cabinet in that location if one doesn't exist. Is there an operation that lets me get a ...