The tag has no wiki summary.

learn more… | top users | synonyms

40
votes
5answers
5k views

Why can't RDBM's cluster the way NoSQL does?

One of the big plusses for nosql DBMS is that they can cluster more easily. Supposedly with NoSQL you can create hundreds of cheap machines that store different pieces of data and query it all at ...
12
votes
3answers
5k views

What does horizontal scaling mean?

In database context, I have come across horizontal scalability as one of the advantages of the NOSQL databases. What does the term mean? http://en.wikipedia.org/wiki/CouchDB ...
7
votes
3answers
1k views

SQL Server - Multiple running totals

I've got a base table with transactions and I need to create a table with running totals. I need them to be per account and and also have a few running totals for each account (depending on the ...
7
votes
3answers
1k views

Testing stored procedure scalability

I have an email application that will be called upon to deliver to the UI the number of new messages for a given user on each page load. I have a few variations of things I am testing on the DB level ...
6
votes
3answers
2k views

Database suggestion for a social network/ knowledge base community?

I am looking into various database types and DBMS's for a new project I am wanting to start in the summer. I have built systems in MySQL and postgreSQL , now I am wanting to expand my knowledge and ...
5
votes
3answers
5k views

Why can't relational databases meet the scales of Big Data?

It is often repeated that the Big Data problem is that relational databases can not scale to process the massive volumes of data that are now being created. But what are these scalability limitations ...
5
votes
1answer
4k views

Scalability limitations of PostgreSQL and MySQL

I've heard that performance of non-sharded relational database such as MySQL or PostgreSQL "breaks" beyond 10 TB. I suspect that limits as such do exist as one would not come up with Netezza, ...
5
votes
3answers
489 views

How to store many smallish time-series in a relational DB?

Input Data Multiple Tests Beds generate measurement data of various complexity. In it's most basic form, not considering any meta-data, one measurement on a Test Bed will be a small (1 - a few ...
4
votes
3answers
1k views

MySQL tables/databases into different disks

I am aware that there is a possibility to have a single database server with many tables/databases, and you can separate some of those tables/databases into separate disks. Lets say you have 5 ...
4
votes
4answers
233 views

In Memory Database for high transaction and volume spikes on a website [closed]

I am looking at the IA for a hypothetical application scenario, for example: A high demand application such as an online ticketing website where there can be demand for 100,000 tickets to be ...
4
votes
3answers
4k views

How does one scale SQL Server 2008 or 2012?

How does one scale SQL Server 2008 (or 2012)? At its basic, I understand there are two options: Scale up: If CPU bound, I can clearly see going from 1 CPU core to 2 to 4. Or if RAM usage rockets, ...
4
votes
1answer
532 views

Postgres scalability - What is the impact of connection pooling?

We had a question over on Server Fault that raised an interesting question: In light of the scalability improvements in Postgres 9.2 scalability improvements, is it better to use connection pooling ...
4
votes
2answers
1k views

Proper storage size estimation technique for PostgreSQL

We are preparing a PostgreSQL database for production usage and we need to estimate the storage size for this database. We are a team of developers with low expertise on database administration, so we ...
4
votes
1answer
722 views

Database as a service (DbaaS) over dimensions--performance, scalability, & reliability?

MySql Database as a service: Amazon Xeround Another eg. PostgreSQL Database-as-a-Service. You can get the list of similar products here. Does anyone have details on their performance, ...
4
votes
1answer
1k views

What are the practical limitations on a column family in Cassandra?

In Cassandra, it's not recommended to have more than a few thousand column families, let's say 2,000 for the sake of argument. In cases where more than 2,000 types of data need to be persisted, one ...
4
votes
3answers
920 views

Are there any in-memory NoSQL databases?

Given that NoSQL databases are designed to support horizontal scalability by throwing in more servers, are there any that are designed to do entirely without persistent disk storage? I'm looking at ...
3
votes
3answers
2k views

Is it better to seperate a big query into multiple smaller queries?

There are situations which require to have really big query joining several tables together with sub select statements in them to produce the desired results. My question is, should we consider using ...
3
votes
1answer
225 views

to what “scale” of applications does nosql support?

Recently i came across the Nosql database concept, though i learnt how to and why of it, i did not get a proper answer to the question , to what scale of project does it support? Will it support ...
3
votes
2answers
154 views

Scalability: Desiging a message table

This question might be deadly simple but I wanted to make sure I am going through the correct path and not making any simple and irrevertible mistakes. My project will be based on messaging and it ...
3
votes
2answers
532 views

Scalable DBaaS that can scale to hundreds of terabytes with easy backup and restore

I've gone over many DBaaS solutions, including Xeround (limited to 50G, in memory), NuoDB (currently in beta 7), DrawnToScale (not out yet), MongoDB (in memory), RavenDB (for ASP.NET, but NoSQL) and ...
3
votes
1answer
408 views

NuoDB with Amazon S3

I think about developing my new mobile and web app against NuoDB. I believe in their technology. The thing that I care the most is backup and restore in case of data corruption and read / write speed ...
3
votes
1answer
129 views

Confused about load balancing and horizontal scaling in postgresql [closed]

Please correct me if I am wrong but I guess handling more requests and load by adding more machines or balancing the load between multiple servers is horizontal scalling. So, if I add more servers, ...
3
votes
1answer
355 views

PostgreSQL High Availability/Scalability using HAProxy and PGBouncer

I have multiple PostgreSQL servers for a web application. Typically one master and multiple slaves in hot standby mode (asynchronous streaming replication). I use PGBouncer for connection pooling: ...
3
votes
1answer
755 views

Very Slow MySQL Queries, even with indexes

I have a relatively large 4-deep relational data setup like this: client_applications: (potentially 1,000's of records)    - ...    - account_id    - ...
3
votes
0answers
206 views

SQL Server 2008 R2 - Merge Replication - Can the Distribution Database size cause replication problems?

Setup: We have a SQL Server 2008 R2 instance running on a central server. The server is a merge-replication publisher. We have 'N' subsribers running SQL Server 2008 Express. 'N' can grow to any ...
2
votes
1answer
173 views

How to scale MySQL transaction wrapped queries?

So I'm working on an app that wraps many of it's SQL queries in a transaction for it's SQL usage and I'm trying to think of some ways to scale the DB layer up.... Up until now we've been throwing ...
2
votes
2answers
158 views

A Huge Database: Picking the right schema and data model

We want to store some genomic variant data but there are some problems, more important ones like problem of the data's immense size and variability. 1) Variant data can be huge. For example, a single ...
2
votes
2answers
428 views

Is it OK to create hundreds of databases in SQL Azure versus one big one and run the risk of deadlocking

I need to create highly scalable solution - where field devices in thousands of sites are delivering data in real time to a back end system, and SQL Azure seems to fit the bill nicely in terms of ...
1
vote
2answers
1k views

Stress test MySQL with queries captured with general log in MySQL

Is there any tool available which can do stress testing using the log file created by MySQL general log? After a lot of search on google I found few stress testing tools which only use some benchmarks ...
1
vote
4answers
464 views

database sharding on mysql

I need to do database sharding(horizontal) based on MySQL. My database schema is as following: I have tables A,B,C A,B is Global(which need not to be sharding) C have huge data so it need to be ...
1
vote
2answers
835 views

MySQL Cluster On Cloud vs MySQL Cluster On Dedicated Cluster

I saw this interesting article : http://mikaelronstrom.blogspot.com/2012/02/105bn-qpm-using-mysql-cluster-72.html and it made me get interested in MySQL Cluster. After little chat with hosting ...
1
vote
1answer
271 views

Mitigating MySQL replication delay in single master/multiple slave setup

For a particular web application that requires high availability I am thinking of scaling out by setting-up MySQL replication with one "write" master and multiple "read" slaves (as explained here ...
1
vote
2answers
102 views

How to store massive quantities of machine generated data?

I am tasked with developing a system that stores massive quantities of machine generated data (syslogs) from a number of server farms and I am wondering what tools you fine folks use such instances. ...
1
vote
2answers
601 views

Which is better for long term database scalability: adding columns or having key/value store

For application, which has few (5-10) settings at the moment but will have a lot more (up to 100) in the future which would be better approach: to add new columns to settings table every time new ...
1
vote
1answer
260 views

MySQL table locks solution -> InnoDb / Partitions

I'm experiencing many table locks in a MyISAM MySQL table that is pretty big (Constx10xGB and 15M rows). It's mainly caused due many heavy inserts. My current thoughts to optimize: Changing to ...
1
vote
1answer
1k views

64 bit Postgres 9.1 Limitions on Windows 2008 R2?

When Deploying Postgres on Windows Server 2008 R2: Are there any limitations on windows that don't exist on Linux? Does the postgres scale in the same way on windows as it does on Linux? Is ...
1
vote
0answers
69 views

PostgreSQL how to create a scalable location-based query

I need suggestions because I am not good enough. I have a DB in PostgreSQL that runs on AWS (Amazon Web Services). I have a table "user_location" where there are stored the locations of each user ...
1
vote
1answer
212 views

optimizing MySQL for traffic analytics system

background : I've developed a URL shortener system like Bitly with same features , so the system also tracks clickers info and represent as graphs to the person who has shorten the link as ...
1
vote
1answer
547 views

What are the pros and cons of using CLIENT_RESULT_CACHE_SIZE and RESULT_CACHE_MODE

I have multiple questions For Oracle Database 11g Release 11.2.0.1.0 what are the benefits of using CLIENT_RESULT_CACHE_SIZE and RESULT_CACHE_MODE. How will the CLIENT side cache be kept sync if ...
0
votes
1answer
85 views

Looking for a scalable relational database [closed]

I'm looking for a database that achieves the following: fully ACID. relational with joins. has transactions. allows the administrator to specify sharding rules (for example: users from Europe should ...
0
votes
1answer
79 views

Scaling out mongodb on the same storage

We are planning to have some tests to predict how mongo db operates and functions with 2 terabyte of inforamtion on a single collection. So the plan is to have 15-20 sharded machines (on the same ...
0
votes
1answer
182 views

Multiple Partition Keys?

This is a follow-on from a previous question. I've never worked with SQL Server partitioning but I currently faced with designing a database for which the volumes probably warrant it. The system is ...
0
votes
0answers
10 views

Choosing In-memory data grid / KV store for C++ project, targeting high throughput and elastic failover

(question moved from SO) I'm looking for IMDG/KV solution matching following criteria: must be distributed/HA: masterless/multi-write-master wanted has C/C++ API max. 200 microseconds latency ...
0
votes
0answers
59 views

MySQL: Optimizing for large but discrete data sets

In brief: I'm developing a database that handles GTFS datasets from multiple transit agencies. Each dataset contains millions of rows in the stop_times.txt file (and thus its corresponding table). ...
0
votes
0answers
83 views

MySql Scalability, Master Slave replication a good idea? Read description please

I have a database on mysql which is scaling on daily basis and right now its on 5M records and in a month it may double itself. I have a table containing users data which is the largest one. I have ...
-3
votes
1answer
71 views

Are database servers less scalable than web servers? [closed]

Isn't it possible to scale database servers through virtualization, like forever? I like to keep most of the workload on database server because it optimizes queries and the architecture decreases the ...