This is a question for DBAs with a lot of practical experience with large scale deployments.
We are trying to build an app with very low latency. (We really want to create the best user experience, and latency matters in this case.)
The app is fundamentally a RESTful Node.js server that queries a database (OrientDB). Static content/files are hosted elsewhere. Objects are translate into UI largely on the client end.
We are planning on using Amazon EC2 to host the web server and database.
High availability is important as well.
Cost is not a major deciding factor at this time.
What architecture would you recommend?
Here are 2 options, maybe there's more:
Put the webserver (Node.js in this case) on the same EC2 instance as the database. We'd have multiple instances like this for availability and load balancing. In the past, I've used this to reduce latency in high volumes but those system didn't have to scale as dramatically. It was also before placement groups.
Or, have databases on different instances than the REST web server.
Background
The database size is about 50 GB and we need a better solution. Projections look like traffic will be 10x that amount (a 500 GB DB) within 8 months.
It is about 20% writes and 80% reads. There's about 1000 queries per second right now and the rate is increasing.