We are in the process of putting together a whole system for a new startup company. This project is meant to store thousands of requests and each one needs to be stored and displayed in real-time.
This data will then be analysed so as to gather insight on traffic patterns etc.
After conducting a good amount of research we thought that HBase would be a perfect fit due to its HDFS replication, MapReduce via Hadoop and fast read/write mechanism. However the issue is that it is recommended to have a couple of million rows when implementing Hbase for the first time.
Obviously being a startup we would not have such traffic in the beginning however the number of requests is expected to increase quite quickly over a short period of time.
Therefore what would you suggest, still going for HBase or starting with e.g. MySQL and change once we have the right amount of traffic? The issue with using MySQL or any other DBMS is the aspect of real-time whilst also having to shift the whole system over to HBase should we start running into issues of sharding etc.