The tag has no wiki summary.

learn more… | top users | synonyms

44
votes
5answers
7k views

Why can't RDBM's cluster the way NoSQL does?

One of the big plusses for nosql DBMS is that they can cluster more easily. Supposedly with NoSQL you can create hundreds of cheap machines that store different pieces of data and query it all at ...
13
votes
3answers
6k views

What does horizontal scaling mean?

In database context, I have come across horizontal scalability as one of the advantages of the NOSQL databases. What does the term mean? http://en.wikipedia.org/wiki/CouchDB ...
9
votes
3answers
2k views

Testing stored procedure scalability

I have an email application that will be called upon to deliver to the UI the number of new messages for a given user on each page load. I have a few variations of things I am testing on the DB level ...
9
votes
3answers
3k views

Database suggestion for a social network/ knowledge base community?

I am looking into various database types and DBMS's for a new project I am wanting to start in the summer. I have built systems in MySQL and postgreSQL , now I am wanting to expand my knowledge and ...
8
votes
3answers
8k views

Why can't relational databases meet the scales of Big Data?

It is often repeated that the Big Data problem is that relational databases can not scale to process the massive volumes of data that are now being created. But what are these scalability limitations ...
8
votes
3answers
2k views

SQL Server - Multiple running totals

I've got a base table with transactions and I need to create a table with running totals. I need them to be per account and and also have a few running totals for each account (depending on the ...
6
votes
1answer
7k views

Scalability limitations of PostgreSQL and MySQL

I've heard that performance of non-sharded relational database such as MySQL or PostgreSQL "breaks" beyond 10 TB. I suspect that limits as such do exist as one would not come up with Netezza, ...
5
votes
3answers
4k views

Is it better to separate a big query into multiple smaller queries?

There are situations which require having really big query joining several tables together with sub select statements in them to produce the desired results. My question is, should we consider ...
5
votes
1answer
2k views

What are the practical limitations on a column family in Cassandra?

In Cassandra, it's not recommended to have more than a few thousand column families, let's say 2,000 for the sake of argument. In cases where more than 2,000 types of data need to be persisted, one ...
5
votes
3answers
849 views

How to store many smallish time-series in a relational DB?

Input Data Multiple Tests Beds generate measurement data of various complexity. In it's most basic form, not considering any meta-data, one measurement on a Test Bed will be a small (1 - a few ...
4
votes
3answers
2k views

MySQL tables/databases into different disks

I am aware that there is a possibility to have a single database server with many tables/databases, and you can separate some of those tables/databases into separate disks. Lets say you have 5 ...
4
votes
4answers
524 views

In Memory Database for high transaction and volume spikes on a website [closed]

I am looking at the IA for a hypothetical application scenario, for example: A high demand application such as an online ticketing website where there can be demand for 100,000 tickets to be ...
4
votes
3answers
5k views

How does one scale SQL Server 2008 or 2012?

How does one scale SQL Server 2008 (or 2012)? At its basic, I understand there are two options: Scale up: If CPU bound, I can clearly see going from 1 CPU core to 2 to 4. Or if RAM usage rockets, ...
4
votes
1answer
884 views

Postgres scalability - What is the impact of connection pooling?

We had a question over on Server Fault that raised an interesting question: In light of the scalability improvements in Postgres 9.2 scalability improvements, is it better to use connection pooling ...
4
votes
2answers
2k views

Proper storage size estimation technique for PostgreSQL

We are preparing a PostgreSQL database for production usage and we need to estimate the storage size for this database. We are a team of developers with low expertise on database administration, so we ...
4
votes
1answer
758 views

Database as a service (DbaaS) over dimensions--performance, scalability, & reliability?

MySql Database as a service: Amazon Xeround Another eg. PostgreSQL Database-as-a-Service. You can get the list of similar products here. Does anyone have details on their performance, ...
4
votes
3answers
985 views

Are there any in-memory NoSQL databases?

Given that NoSQL databases are designed to support horizontal scalability by throwing in more servers, are there any that are designed to do entirely without persistent disk storage? I'm looking at ...
3
votes
1answer
257 views

to what “scale” of applications does nosql support?

Recently i came across the Nosql database concept, though i learnt how to and why of it, i did not get a proper answer to the question , to what scale of project does it support? Will it support ...
3
votes
2answers
190 views

Scalability: Desiging a message table

This question might be deadly simple but I wanted to make sure I am going through the correct path and not making any simple and irrevertible mistakes. My project will be based on messaging and it ...
3
votes
2answers
637 views

Scalable DBaaS that can scale to hundreds of terabytes with easy backup and restore

I've gone over many DBaaS solutions, including Xeround (limited to 50G, in memory), NuoDB (currently in beta 7), DrawnToScale (not out yet), MongoDB (in memory), RavenDB (for ASP.NET, but NoSQL) and ...
3
votes
1answer
3k views

PostgreSQL High Availability/Scalability using HAProxy and PGBouncer

I have multiple PostgreSQL servers for a web application. Typically one master and multiple slaves in hot standby mode (asynchronous streaming replication). I use PGBouncer for connection pooling: ...
3
votes
1answer
442 views

NuoDB with Amazon S3

I think about developing my new mobile and web app against NuoDB. I believe in their technology. The thing that I care the most is backup and restore in case of data corruption and read / write speed ...
3
votes
1answer
994 views

Confused about load balancing and horizontal scaling in postgresql [closed]

Please correct me if I am wrong but I guess handling more requests and load by adding more machines or balancing the load between multiple servers is horizontal scalling. So, if I add more servers, ...
3
votes
1answer
2k views

Very Slow MySQL Queries, even with indexes

I have a relatively large 4-deep relational data setup like this: client_applications: (potentially 1,000's of records)    - ...    - account_id    - ...
3
votes
0answers
242 views

SQL Server 2008 R2 - Merge Replication - Can the Distribution Database size cause replication problems?

Setup: We have a SQL Server 2008 R2 instance running on a central server. The server is a merge-replication publisher. We have 'N' subsribers running SQL Server 2008 Express. 'N' can grow to any ...
2
votes
4answers
124 views

What are the risks of building a database outside IT control?

I've been looking for the best StackExchange community to ask this. I hope it fits well here. As a risk analyst in a trading business, I'm often confronted with large volumes of data, way beyond the ...
2
votes
1answer
350 views

SQL Server many-to-many relationship, row reuse?

Below is your typical many-to-many relationship. Pretend there are 2 tasks that have the exact same Tags. because of foreign key/primary key relationship constraints, there is no way for the rows in ...
2
votes
1answer
238 views

How to scale MySQL transaction wrapped queries?

So I'm working on an app that wraps many of it's SQL queries in a transaction for it's SQL usage and I'm trying to think of some ways to scale the DB layer up.... Up until now we've been throwing ...
2
votes
2answers
169 views

A Huge Database: Picking the right schema and data model

We want to store some genomic variant data but there are some problems, more important ones like problem of the data's immense size and variability. 1) Variant data can be huge. For example, a single ...
2
votes
2answers
589 views

Is it OK to create hundreds of databases in SQL Azure versus one big one and run the risk of deadlocking

I need to create highly scalable solution - where field devices in thousands of sites are delivering data in real time to a back end system, and SQL Azure seems to fit the bill nicely in terms of ...
2
votes
1answer
37 views

How do you generate large OLTP and reporting workloads? [closed]

My current skillset lies towards plain vanilla data warehousing, with < 20k users, our setups are rather simple.. I am looking for ways to enhance my technical skills, however, as much theory as I ...
1
vote
2answers
2k views

Stress test MySQL with queries captured with general log in MySQL

Is there any tool available which can do stress testing using the log file created by MySQL general log? After a lot of search on google I found few stress testing tools which only use some benchmarks ...
1
vote
2answers
861 views

Which is better for long term database scalability: adding columns or having key/value store

For application, which has few (5-10) settings at the moment but will have a lot more (up to 100) in the future which would be better approach: to add new columns to settings table every time new ...
1
vote
4answers
560 views

database sharding on mysql

I need to do database sharding(horizontal) based on MySQL. My database schema is as following: I have tables A,B,C A,B is Global(which need not to be sharding) C have huge data so it need to be ...
1
vote
2answers
1k views

MySQL Cluster On Cloud vs MySQL Cluster On Dedicated Cluster

I saw this interesting article : http://mikaelronstrom.blogspot.com/2012/02/105bn-qpm-using-mysql-cluster-72.html and it made me get interested in MySQL Cluster. After little chat with hosting ...
1
vote
1answer
503 views

Mitigating MySQL replication delay in single master/multiple slave setup

For a particular web application that requires high availability I am thinking of scaling out by setting-up MySQL replication with one "write" master and multiple "read" slaves (as explained here ...
1
vote
2answers
116 views

How to store massive quantities of machine generated data?

I am tasked with developing a system that stores massive quantities of machine generated data (syslogs) from a number of server farms and I am wondering what tools you fine folks use such instances. ...
1
vote
1answer
29 views

Denormalization Strategies: When is it appropriate to create redundancy for faster querying?

So I sometimes see databases purposely add redundant information for faster querying. I saw that this was mentioned here but I was wondering when would this be better than indexing, or creating a ...
1
vote
1answer
48 views

Reference implementation of a scalable storage system with strong consistency?

Is there a storage system with these properties? horizontally scalable - can handle arbitrary read and write throughput if you add enough nodes to the cluster consistent, in the sense that all reads ...
1
vote
1answer
336 views

MySQL table locks solution -> InnoDb / Partitions

I'm experiencing many table locks in a MyISAM MySQL table that is pretty big (Constx10xGB and 15M rows). It's mainly caused due many heavy inserts. My current thoughts to optimize: Changing to ...
1
vote
1answer
1k views

64 bit Postgres 9.1 Limitions on Windows 2008 R2?

When Deploying Postgres on Windows Server 2008 R2: Are there any limitations on windows that don't exist on Linux? Does the postgres scale in the same way on windows as it does on Linux? Is ...
1
vote
1answer
28 views

Are there any inherently non-partitionable datasets?

Are there any practical datasets that, by their nature, cannot be partitioned in any practical way? I am wondering about this because such datasets would be ill-suited to scaling out on commodity ...
1
vote
0answers
36 views

Modulus calculation for compound primary keys in MySQL Cluster (and other sharded databases)

In MySQL Cluster, and generally in sharded databases, how is a compound primary key's modulus calculated? Say, for example, there are two tables that have primary keys like so: CREATE TABLE Orders ...
1
vote
0answers
131 views

PostgreSQL how to create a scalable location-based query

I need suggestions because I am not good enough. I have a DB in PostgreSQL that runs on AWS (Amazon Web Services). I have a table "user_location" where there are stored the locations of each user ...
1
vote
1answer
336 views

optimizing MySQL for traffic analytics system

background : I've developed a URL shortener system like Bitly with same features , so the system also tracks clickers info and represent as graphs to the person who has shorten the link as ...
1
vote
1answer
1k views

What are the pros and cons of using CLIENT_RESULT_CACHE_SIZE and RESULT_CACHE_MODE

I have multiple questions For Oracle Database 11g Release 11.2.0.1.0 what are the benefits of using CLIENT_RESULT_CACHE_SIZE and RESULT_CACHE_MODE. How will the CLIENT side cache be kept sync if ...
0
votes
1answer
93 views

Looking for a scalable relational database [closed]

I'm looking for a database that achieves the following: fully ACID. relational with joins. has transactions. allows the administrator to specify sharding rules (for example: users from Europe should ...
0
votes
1answer
75 views

Optimization for 100 GB Real-time info table

I have a table of device information feed which is filled by data from remote device units. There will be like 1000 units each sending an update per minute which will be stored as a row in that ...
0
votes
1answer
117 views

Scaling out mongodb on the same storage

We are planning to have some tests to predict how mongo db operates and functions with 2 terabyte of inforamtion on a single collection. So the plan is to have 15-20 sharded machines (on the same ...
0
votes
1answer
196 views

Multiple Partition Keys?

This is a follow-on from a previous question. I've never worked with SQL Server partitioning but I currently faced with designing a database for which the volumes probably warrant it. The system is ...