I'm working on a geo-replicated web platform, composed of an ecosystem of microservices. We need to improve and rework the user activity tracing pipeline and I'm looking for "the best" database to achieve that.
Our platform relies entirely on kubernetes, for that reason we would exclude every technology that is not compatible with this approach.
Each log is quite simple and potentially made of the following data:
- timestamp
- user_id
- action_type
- description
- some metadata, their format can be adapted to choice of the database (json, key-value, and so on)
Goals
- high available write operation
- excellent write-scale capability
- excellent storage scalability
Plus
- geo-replication
- cloud native
- hot-warm architecture/some kind of rotation
Non goals
- complex data aggregation
- complex search query
- batch processing
Based on my knowledge, researches and experience, good candidates would be:
- cassandra: should satisfy all the goal and geo-replication
- cockroach: I've never used it before, but based on the documentation all the goals should be satisfied + geo-replication and is cloud native
- influxDB: Not sure about that, I've been using influxdb for a while and though it should satisfy all the goals and all the plus, maybe is not the best choice for this kind of data
what I would not choose:
- elasticsearch: it does a lot of things I don't need, is tricky to be maintained and set up, is very resource-consuming
- mongodb: the write scalability can be achieved only with sharding, this configuration is hard to be maintained and evolved, the shard key is tricky to be changed. Not fully HA due to the master-slave election mechanism
- all the classic SQL with a single master
UPDATE:
good candidate for functionality but is a monster (and I don't know how it works with kubernetes)
- HDFS
