We are a MySQL front-end shop and have a system currently logging event data in a highly structured format. Think apache traffic log.
We need to be able to aggregate counts of this data (cube) vis a vi ad-hoc queries. We are currently sending the data to CouchDB. This is very fast, once a view has mapped completely, but our database size is only 31GB and CouchDB is taking FOREVER to map a new view ( 2+hours ). Retrieving JSON formatted data has worked well for us. But we dread creating new view documents. I'm thinking all Map-Reduce systems may have this problem.
We are evaluating Infobright as they claim to provide superior aggregation performance for ad-hoc queries and are a MySQL fork.
We have evaluated Mondrian and it's not going to work for us.
MySQL/InnoDB is too slow.
We will be around 500GB of log data by the end of the year.
Is Infobright the right solution or should we be also evaluating something else?