Splitting a database table into multiple segments for performance or manageability.

learn more… | top users | synonyms (1)

19
votes
2answers
4k views

What is the modern way to partition PostgreSQL across machines, when the data is “naturally partitionable”

After several years of dwelling into the "NoSQL" space, now I have a problem that is quite "relational" in its nature. Today I see data stores with quite different eyes than before. Things like Riak ...
1
vote
2answers
1k views

Will enabling row movements on list-partitioned table cause performance problems(oracle 11g r2)

Our incident management system has run for years, and the large quantity of closed incident tickets makes the table rather huge and slows down the search queries. Our database version is oracle 11g ...
3
votes
1answer
277 views

Adding new key(s) to enable partitioning, INSERT performance?

I have a table to storage GPS coordinates with multiple (~30) inserts per second (from 50 different devices) during 12 hours per day, so the table is growing fast, to improve the SELECT speed I was ...
0
votes
3answers
286 views

Active/current (1-5%) and historical records in same table: index or partitioning?

I am trying to discover the correct architecture for the following situation. An application stores 'source data' in multiple tables. Each of these 'source data' tables contains actual, current, ...
1
vote
1answer
41 views

How to efficiently swap a portion of a large table's data?

I run an application (price comparison website) where I need to update a pricing table with dozens of millions of rows every day. Basically, every product feed I download from a merchant results in ...
2
votes
1answer
396 views

Script Table Definition in Powershell

I'm writing a powershell to create a duplicate table(in another schema) for the purposes of partition switching*. I'm having problems adding the columns and the clustered column store index. Here's ...
1
vote
0answers
108 views

MySQL table partition strange behavior (slow query suddenly)

(I asked this question in "stack overflow" and was recommended to ask in here). (MySQL version: 5.6.15) I have a huge table (Table_A) with 10M rows, in entity-attribute-value model. It has a ...
0
votes
3answers
461 views

The column order of composite clustered primary key on a partitioned table? [closed]

I have the following table. create table T (K1 date, K2 int, ....) on partitionScheme(K1) The table will be partitioned by K1 (K1 has low selective. the data will be appended in the order by K1). ...
2
votes
2answers
185 views

Partition : understanding files and file groups

We are trying to partition a large history table. Being new to partitioning, I was trying to understand the concept of files and file groups. File groups can be set up in following three ways : 1. ...
0
votes
0answers
385 views

Using partitioned table, and dealing with auto_increment PK

I have a big table (>100 million rows) that tracks user's activities. It has an ID, which is PK and has auto_increment property. This is the only unique key. Table also has userid and datetime column. ...
3
votes
1answer
569 views

SQL Server data files in Windows Azure: filegroups and partitioning still applicable?

I just created a database using the new functionality "SQL Server 2014 data files on Windows Azure" and this works quite well. However, I have a table with a billion records and in our local ...
0
votes
0answers
62 views

Oracle estimate cardinality on partitioned table

How does Oracle estimate cardinality on a partitioned table if the query gets the list of partitions from a subquery or a join? For example, if I have a query like this select * from ...
1
vote
1answer
115 views

Getting rid of partitioning in Postgresql 9

it turns out that our approach with list partitioning was not the best idea. We have way to much small partitions and a lot of time is wasted on partition choosing by query optimizer. What's the ...
0
votes
1answer
62 views

Help with optimising a sql query on two partitioned tables

I have a SQL query that I am running on two very large tables in SQL Server 2012 Enterprise The two tables are ACKs and Logs. Both are partitioned on their time by day column with the appropriate ...
2
votes
2answers
125 views

Make sure partitioning is working

I created a partition for a table. There is a column "archived" (tinyint). 0 goes to PRIMARY data file, 1 to ARCHIVED data file. ALTER DATABASE MYDB ADD FILEGROUP [ARCHIVED]; ALTER DATABASE MYDB ADD ...
2
votes
2answers
252 views

Update on view over partitioned tables updating all indexes

We have a table, which is partitioned through a date field into separate years. There is a view over all of these tables (Call) Schema is as follows: CREATE TABLE [dbo].[Call_2015]( [calID] ...
1
vote
0answers
53 views

SQL Server partitioning of many-to-many tables - triggers

Many to many tables usually have only foreign keys to related tables, like EmployeeId PositionId Is it good idea to add column only for partitioning and set its value with INSTEAD OF INSERT/UPDATE ...
0
votes
0answers
197 views

Performance degradation after partioning tables using DBMS_REDEFINITION package

We have created partitions on per-existing tables in my application using DBMS_REDEFINITION package. just for the sake of improving performance. Now , We are facing an issue in the performance ...
2
votes
1answer
187 views

Adding Indexes To a Partitioned table in SQL Server 2012

I have recently converted our one large table to use table partitioning in SQL Server 2012. It's a table that stores logs based on a time. So I created a partition function to split the table on my ...
0
votes
0answers
287 views

Renaming the system generated partitions during interval partitioning

I am trying to rename the system-generated partition names (generated during interval partitioning). I try to do this while my table is live(data read and write is happening). I need two scripts. I ...
0
votes
1answer
1k views

Efficient way to DELETE whole partition's data in table partitioned by HASH(DAY(datetime))?

I have a MySQL InnoDB table PARTITIONed by HASH(DAY(dateTime)). I'd like to remove partitions holding data older than 10 days. CREATE TABLE `partitioned` ( `id` bigint(20) unsigned NOT NULL ...
0
votes
0answers
26 views

How to figure out how to split a table using the proper scheme

If you have a big table that you want to partition and you have 2 candidates as the partitioning key (in the sense that 50% of the queries use one column and 50% of the queries use the other column ...
2
votes
1answer
128 views

How does subpartitioning actually work and what physical files are created?

I can't understand how the subpartitioning works in mysql. I am reading the following example: create table invoices ( customer_id int not null, product_id int not null, amount decimal ...
4
votes
1answer
9k views

Join tables in two databases make query slow? is better to partition db?

I have a table1 in db1 and table2 in db2 on SQL Server 2008 R2. If I make a select query joining both tables, it's really slow to get the results. A simple query like SELECT * FROM db1.dbo.table1 ...
1
vote
2answers
6k views

Dynamic Table partitioning on a daily basis

I have a SQL Server database which contains two tables -- Acks and Logs. These two tables are related in logic but not in a relational database way. Basically, every message that comes in gets saved ...
3
votes
2answers
4k views

Get all partition names for a table

I want to list all the partitions created by dynamic triggers in PostgreSQL 9.1. I was able to generate a count of partitions using this related answer by Frank Heikens. I have a table foo with an ...
0
votes
3answers
65 views

Redesign basic Mysql storage architecture

We are a small Internet provider and we are storing all our clients traffic data in a MySQL database (MyISAM). We insert data every few minutes in three tables like this: capture_table ---------- ...
1
vote
1answer
179 views

Archive partition before delete it?

I have manage to create an event and with the following scripts could both create and delete the partition. The issue now I need help on how to save the partition before its deleted? Should I use ...
0
votes
0answers
46 views

Adding Partitions to an Existing Filegroup

For maintenance reasons, the organization that controls the servers where I work has decided to limit the number of file groups and files for partitioning to just 1 of each. Their reason for this is ...
5
votes
2answers
3k views

Partitioning a Table Referenced by a Foreign Key

I have one table: CREATE TABLE [dbo].[entry] ( [ID] [bigint] IDENTITY(1,1) NOT NULL, [EntryDate] [datetime] NOT NULL, [createddate] [datetime] NOT NULL, CONSTRAINT [PK_Entry_ID] ...
1
vote
0answers
164 views

Is it best practice to include partition keys within foreign keys when referencing partitioned tables?

The large tables in our database are partitioned by an id in order to separate data and speed up IO. The table of the data is partitioned by the partition key, but the primary key is only on an ...
3
votes
2answers
290 views

mysql - is it possible to create a partition expression based on two keys / indexes?

if i have a table with, lets say 500 million rows, and among others there are two main individual indexes on that table. the table looks like: CREATE TABLE `t1` ( id_1 bigint unsigned not null, ...
2
votes
1answer
29 views

Obtaining the next partition number

For a partitioned table, how do I determine via T-SQL what is the next partition number that will be used? For example, the switch-in today went into partition number 50. I want to build the new ...
1
vote
1answer
1k views

MySQL 5.6.17 running out of memory when opening table with many partitions

We are currently setting up a MySQL database (version 5.6.17) on an Amazon db.m1.large RDS instance for storing time series data. Each time series property of an object is stored in a separate InnoDB ...
0
votes
0answers
346 views

Copying a table with its structure, indexes, and associated triggers?

I have studied this post on Stack Overflow: Fastest way to copy a table in MySQL. But no answer is copying everything like the table's structure, data, indexes as well as associated triggers. Is it ...
0
votes
0answers
148 views

How do triggers work on partitioned tables?

I have partitioned a table on the basis of a column. This column can either have 0 or 1. So, only two partitions (p0, p1) have been created internally. On this table there are 3 triggers which execute ...
0
votes
2answers
495 views

SUM() the result of a stored procedure

I use the Pl/Proxy extension in order to shard a table. It works like a charm, except for this : I have a stored procedure which runs on 4 shards. It looks like this : CREATE OR REPLACE FUNCTION ...
2
votes
1answer
267 views

Is there a difference between RANGE and LIST method for partitioning by consecutive integer values?

I have a table with a year column that has consecutive values -- 2000, 2001, and so on. It's a read-heavy MyISAM table, with 25+ million rows for every value of year and a comparable set inserted in ...
1
vote
1answer
278 views

Paritioning performance for full table scan on non-paritioned index

We have a table in Postgres that we are considering partitioning. We occasionally query an indexed field that is separate from one with the partition key. My question is: Can partitioning the table ...
1
vote
0answers
340 views

SQL Server : ALTER TABLE tablename SWITCH PARTITION 2 to tempempty

Have a sliding partition where I need to remove the trailing partition when they expire. I create an empty table that matches the existing table. When I create the empty table, I specify the file ...
4
votes
4answers
255 views

Are we a good fit for SQL Server Enterprise and table partitioning?

We currently have a production database that contains millions of records (14 million in the largest table). One function of the database is to provide time sheet information so a single table could ...
1
vote
2answers
8k views

How to remove a table partition

I would like to remove a partition from a table in a SQL Server 2012 database. The table has a primary key but the partition column is not a primary key. I would also like to keep the data structure ...
19
votes
3answers
9k views

How Does Table Partitioning Help?

I am having difficulty to grab the idea of pros and cons of table partitioning. I am about to start work on a project which would have 8 tables and one of them will be the main data table which will ...
11
votes
1answer
217 views

How can I interpret the results of these DMVs to help me evaluate our partitioning strategy?

Version: SQL Server 2008 R2 Enterprise Edtn. (10.50.4000) In attempt to evaluate our partitioning strategy, I wrote this query to get the access methods against indexes on partitions (in the broadest ...
2
votes
1answer
825 views

Partitioning MySQL vs PostgreSQL

For our new system we are looking for database server. We considered a lot of pros/cons for MySQL(InnoDB) vs PostgreSQL and the breaking point for us is partitioning. We will choose the one which ...
6
votes
1answer
7k views

ALTER TABLE … SWITCHing from regular table to partitioned table fails

The code below does the following: Creates a database play_partition in C:\TEMP Creates two identical partitioned tables play_table and archive_play_table Switches play_table partition 1 to ...
2
votes
1answer
1k views

Is DBCC SHOWFILESTATS an accurate way to see unused space?

We have a few tables each with near 100 million rows of data. A co-worker has these tables setup with partitions based on a date field. Our disk space is starting to run low and knowing that ...
1
vote
1answer
466 views

PostgreSQL partition pruning?

In postgres partition is done by child tables. My problem are, when we issue a query to master table , does the query optimizer do partition pruning ? i.e select appropriate child table rather than ...
3
votes
1answer
1k views

Partitioning large mysql table

I have a large (40GB) table I would like to partition. The rows are essentially write once read many. Currently I am using mysql 5.5 innodb with file_per_table. It's tough to optimize this table since ...
1
vote
1answer
38 views

Are there any inherently non-partitionable datasets?

Are there any practical datasets that, by their nature, cannot be partitioned in any practical way? I am wondering about this because such datasets would be ill-suited to scaling out on commodity ...