A database system optimised for reporting, particularly in aggregate. Often, but not always implemented using a star schema.

learn more… | top users | synonyms

0
votes
1answer
22 views

data validation Datawarehouse ETL

Need a suggestion from Datawarehouse and ETL experts. I have a project which has bunch of fact and dimension table, and we wire the facts using surrogate keys to dimension (religiously following ...
2
votes
2answers
40 views
+200

Best way to capture historical changes between levels in a dimensional hierarchy

What is the cleanest way to capture changes between levels in a dimensional hierarchy? I have the dimensional hierarchy [Area] > [Region] > [Location], where Area is the parent to Region and Region ...
1
vote
1answer
17 views

Should data warehouse provide data for data composition for real time business transactions [closed]

As title, I always had a feeling that data warehouse should only support for data analysis or the like, but shouldn't be used in real time transactions. (eg. being a data source to provide a product ...
0
votes
0answers
4 views

What is the Difference between Data Transformation vs Transportation?

Is there a difference in meaning between "data transformation" and "data transportation" in ETL process and data warehousing? or both concepts can be used interchangeably in this context? Thanks
0
votes
0answers
6 views

Is REDCap a Data Warehouse Environment?

I working with a web application used to capture research data called The Research Electronic Data Capture (REDCap) system. I am assign to a project to manage the clinical data that are captured in ...
0
votes
1answer
59 views

FK Issue with Fact Table

I have a Datawarehouse star schema. One Dim table is 'DimTweet'. This contains tweet details. For a selected date I want to be able to display the number of tweets. However, I have a problem in my ...
0
votes
1answer
45 views

Convert OLTP database to OLAP Datawarehouse

(http://imgur.com/p8Gy67E). Basically I need help to convert that OLTP Model to Data Warehouse in order to be used for further OLAP Cubes. I understand well the difference between the Star Schema and ...
1
vote
1answer
23 views

Partition Wise Joins in Mysql

Is it possible to create or simulate a partition-wise join in Mysql/Innodb?? Here is a good description of partition-wise joins.... ...
1
vote
1answer
63 views

SQL Server : ETL Stage Index

We have a very large staging table (> 80 GB). From our source system we load invoice data in the staging table. From the staging we transform the data and load into DWH/Fact. Every day we delete the ...
-1
votes
1answer
41 views

Data warehouse backup strategy

I have a data warehouse development environment. Just as the production environment. it loads data every night from test OLTP databases. I want a backup every time the load process finishes. What ...
0
votes
1answer
35 views

ssis surrogate key in parent child tables

I have two tables Person & Ownership having one to many relationship (each person have many ownerships). Primary Key is person_id of type GUID. I am developing SSIS package to load data from both ...
2
votes
1answer
50 views

How to start indexing a new data warehouse

I am building a data warehouse (SQL Server 2012, Kimball Dimensional modeling, 750GB) that will supply data to reports (SSRS), cubes (SSAS) and various datafeeds. I plan to use an index tuning tool ...
1
vote
1answer
77 views

Incremental extract strategy using SSIS and CDC

I'm looking for a strategy for extracting data from a SQL Server database that is configured for change data capture (CDC) using SSIS. Most of the documentation I have seen around using CDC and SSIS ...
4
votes
2answers
94 views

data warehouse automation tool: benefit or bane? [closed]

Today while investigating ETL and data warehouse solutions like Kettle, I came across a new concept (for me) called data warehouse automation tool. I also looked at the white paper here, which ...
6
votes
3answers
291 views

Data warehouse server. How do you calculate RAM/CPU specifications?

I am trying to write a spec for a data warehouse server for our planned data warehouse upgrade. As we run virtual servers on VMWare hosts we have the ability to add or remove resources as necessary. ...
6
votes
1answer
126 views

How do you index alphanumeric foreign keys on large fact tables?

I have a field that is an alphanumeric field that ideally is a encrypted field for a non-unique identifier. It's used to relate other rather large fact tables in a many-to-many relationship. I do not ...
1
vote
2answers
63 views

Ordering of a dimension in sql server datawarehouse

Built and deployed a simple cube with a date dimension. The dimension has things like day of month. How do i sort it? it always shows in the reports as 1, 10, 11, 12, ... 19, 2, 20, 21 Design. ...
6
votes
1answer
82 views

SQL Server Edition for MS 70-463

I just started the MS 70-463 Implementation of warehouse, on the video course the tutor is directing towards SQL Server 2012 Evaluation Edition, but I have already have a SQL Server 2012 Express. Can ...
1
vote
1answer
93 views

SELECT Causes Performance Issue [closed]

I have a SQL SELECT statement that has joins on 27 of our warehouse tables. Original statement was taking more than an hour everyday so I decided to optimize it. I created some indexes but still it ...
1
vote
1answer
45 views

What options are there for rebuilds of indexes on large tables for a data warehouse?

I have a bulk loading process that loads millions of records into a few fact tables within a warehouse. Those tables are primarily clustered on time. The non-clustered indexes are in place for how the ...
0
votes
0answers
33 views

commonly used diagram for ETL

A the title indicates. Am i wondering what is the most commonly used diagram to explain an ETL process? In my case the process is rather linear, would a flowchart suffice? or is a Use-case also ...
1
vote
2answers
59 views

SQL Data warehouse and normalization forms

I don't know if it's a simple question but does a data warehouse have different rules when it comes to normalization forms? At the moment I am creating some tables in a data warehouse, and they ...
5
votes
1answer
237 views

Why should I “Avoid partitioning dimension tables” on SQL Server?

SQL Cat have a list of tips titled Top 10 Best Practices for Building a Large Scale Relational Data Warehouse. Under section 4 - Design dimension tables appropriately they state: Avoid ...
0
votes
0answers
46 views

Which Azure service for use as data warehouse/analysis store?

We currently have a database running in SQL Server 2012 Standard. We need to export data periodically (a "data dump") of a particular set of data to "a location to be decided" where the client's ...
0
votes
2answers
40 views

How can I handle delayed records a month or older on a table clustered on time?

I have a two part question really for anyone working with data warehousing where they have large fact tables. Question 1 Lets say you have a table that has 500 million or more records in it that is ...
0
votes
0answers
22 views

Database choice for structured big data to be queried with SQL

I have a system that harvests and registers a total of 500K (master table) and 100 million (detail table) into a two table master/detail database, per day. The detail table has around 50 fields, two ...
1
vote
0answers
25 views

Which Azure Storage / Database Technology to make use of for large scale logging

I am looking into making use of Azure hosting services in order to develop a logging service for our company. Azure has a number of storage options available (I have listed links below). However I am ...
0
votes
1answer
73 views

Database modelling : Creating a model for restaurant application which gives statistics

I am working on a server-side application in which users will be able to give ratings and comments to different factors of a 'Restaurant'. What the webapp does is takes all the ratings and creates ...
1
vote
2answers
39 views

Should I convert timezones when moving data to a report server/data warehouse

I'm writing an SSIS Package to take data from a production DB to a reports server/data warehouse. The other databases on my reports server are all in Mountain Time but this DB is in UTC should I ...
1
vote
2answers
30 views

Non numeric attributes in fact table (to track data source)?

I am creating a data warehouse using star schemas, following the Kimball methodology. One of the rules is only put numeric values in fact tables. I am considering putting an "id" in a fact table, in ...
1
vote
1answer
39 views

DSV - SSAS Project - New 2 Columns In Table Change More then I Planned

I encountered a very weird thing. I added in my DWH table 2 new columns but when I refreshed the DSV after adding this to my multidimensional project I saw that more (3) columns have been "changed". ...
0
votes
0answers
26 views

Fact Table with day grain that has many records that don't change

So I have a daily snapshot fact table that stores financial account balances. Normally, this would be pretty easy, just have a record for every day for every account. My problem is that table would ...
2
votes
3answers
29 views

Data Warehouse attribute that describes its timeframe?

I am writing a document about a Data Warehouse system, and would like to know if is there a single word (noun) that describes the DW attribute of how many time it contains? I mean, if a DW contains ...
0
votes
1answer
145 views

Time out while connecting to the database. Current time out value is 30 seconds

I am installing a repository creation utility on a windows 8 x64 bit system and getting the following error: Time out while connecting to the database. Current time out value is 30 seconds When ...
1
vote
1answer
75 views

When to run ETL for dimensions

We have a suite of restaurant web applications that stores data in SQL Server 2012 on database server. We have now started to move data for reporting onto a separate data warehouse server. The data ...
0
votes
1answer
123 views

SSAS Calculated Field - rolling sum, for a year of days

I have a fact table storing employee occurrences incurred day over day. I need to have a running sum over the past year of an employee's life. I had, on first pass, done this on ETL, making this ...
1
vote
1answer
28 views

How to know if a particular DW project is using Kimball methodology

When I look at the data warehouse/BI/ETL processes in my group, how can I tell if we are using Kimball methodology or not? If I use Slowly Changing Dimensions i.e expire the old record and create a ...
2
votes
1answer
75 views

Mixed granularity weeks/months time series data in a data warehouse?

I'm not sure that "anti-aliasing of time series data" is the correct terminology to use, so let me explain: I have some sources of data that are aligned quarterly, mostly to do with quarterly running ...
0
votes
0answers
9 views

Pentaho getfilename error

I am try to send 20 mail with attachments using pentahi DI, Jobs are like below Get File Names--->Add constants--->Mail But some mails are not send with below error, Thay say files are not ...
5
votes
1answer
202 views

Time dimension or timestamp in fact table?

Which would you use, and why? A separate time dimension or putting a timestamp in a fact table? Or perhaps both? I am building a data warehouse, and need to represent the time of day that events ...
3
votes
1answer
344 views

How do you calculate how much hardware resources you need for a database?

We are in the process of scaling our database servers. I was wondering how should we calculate how much hardware resources we need for our databases? Here is a little bit information about our ...
3
votes
1answer
43 views

Scripting Microsoft Data Warehouse Wizard

I am trying to automate the MDW setup process in my environment. Currently I have everything complete except for enabling the data collectors (usually uses a wizard as seen ). Has anyone seen or ...
1
vote
1answer
50 views

What's the term that describes our data processes [closed]

We provide an online subscription product that allows our users to research and analyze financial data of their interested industries. As such, we have various tools for loading data from various ...
2
votes
2answers
211 views

When to choose an attribute vs. a new dimension?

I am fully aware of what is a fact, attribute and dimension. However, I'm quite confused to which traits I should choose for dimensions vs attributes of that dimension. For instance, I'm building a ...
0
votes
0answers
38 views

SSAS bridge table with Slowly changing dimensions

I am designing a datamart using the basic principles of Kimball for use in SSAS. I have a many many to many attribute relationships (multi-valued dimension objects), however I have a few questions: ...
3
votes
1answer
123 views

Stored procedure deployment - push or pull into database? [closed]

Using stored procedures as part of an ETL process to populate tables in another database, which method would be considered a best practice (if any) in terms of deployment, maintenance, visibility and ...
0
votes
0answers
39 views

What options exist for replicating SQL Server over the public internet

Specifically, I'd like to set up a reporting copy of a SQL Server 2012 transactional database. This primary runs in a managed services provider, and I want a copy in an AWS EC2 SQL Server instance. ...
0
votes
0answers
75 views

Proxy for MDW data collection on remote servers - SQL Server 2012

I am trying to setup data collection (for disk usage) from remote servers and upload it to my MDW database. I understand I will need a proxy set up for the agent to do this but which subsystem to I ...
14
votes
1answer
415 views

Query strategies using SQL Server 2016 system-versioned temporal tables for Slowly-Changing Dimensions

When using a system-versioned temporal table (new in SQL Server 2016), what are the query authoring and performance implications when this feature is used to handle Slowly Changing Dimensions in a ...
0
votes
0answers
30 views

What is the recommended RAID configuration for data Warehouse? [duplicate]

I am trying to understand if there is RAID best for data warehouse.( for version SQL Server 2012 or 2014). so What is the best RAID for data Warehouse performance wise? What is the recommended RAID ...