Database Administrators Stack Exchange is a question and answer site for database professionals who wish to improve their database skills and learn from others in the community. Join them; it only takes a minute:

Sign up
Here's how it works:
  1. Anybody can ask a question
  2. Anybody can answer
  3. The best answers are voted up and rise to the top

I understand that MySQL and others support large enough data for storing and retrieving. I am also aware of the open source projects such as hadoop and mapreduce etc.. (only their purpose and what they do).

EDIT: when do you bring down the concept of hadoop , pig, mapreduce etc to your application?. Should we use these software in the beginning of the project itself or can it be induced at a later stage after the database is increased to large size?. Any link will be appreciated.

share|improve this question
What do you mean by scaling? Adding hardware? Changing RDBMS software? – JohnFx May 12 '12 at 18:51
it is software , i understand when hardware scaling is done. Am especially focused on scaling using technology like hadoop etc – sree May 12 '12 at 19:35
You scale when the current software isn't meeting demand. Not sure if there is a better answer than that. – JohnFx May 12 '12 at 21:51
@JohnFx i would like to know that demand criteria. i have worked on small projects till date, so i have no idea on what that critical level would be – sree May 13 '12 at 17:33
why down vote? leave a reason atleast !! – sree May 13 '12 at 17:33
up vote 3 down vote accepted

(While the question is broad, I hope that an equally broad answer will spark a useful discussion.)

Please see the accepted answer for What is Hadoop? at SO.

Google searches are a question that Hadoop can answer. Think about the characteristics of a large search engine:

  1. Large amounts of data
  2. Distributed data
  3. Extreme parallelism

Scalability was mentioned in the comments: With Hadoop, it is not hard to throw additional (commodity) servers into the mix.

On to your question. If your project has a lot of SQL and von Neumann bottlenecks, then Hadoop makes little sense. If, however, your data is "Big Data," is less structured, and may be parallelized, then Hadoop will make more sense.

share|improve this answer
Thank you, for your answer appreciate it.. :) – sree Jun 16 '12 at 15:17

Your Answer


By posting your answer, you agree to the privacy policy and terms of service.

Not the answer you're looking for? Browse other questions tagged or ask your own question.