4

I'm building a game that have micro-services structure using docker.

The way i started to build my services is having a mongoDB instance to each service, so having OneToOne service to mongoDB instance,

Resources ---> resourcesDB (mongoDB instance 1)
Chat      ---> chatDB      (mongoDB instance 2)
Buildings ---> buildingsDB (mongoDB instance 3)
Battle    ---> battleDB    (mongoDB instance 4)
...

This really made sense at the beginning as those services for example have nothing in common and don't need have a connection to the same mongo instance, and when they do have something in common (building <-> resources) they just send a call to the needed service and that handles the rest (update his DB).

But i starting to think that this is not the right approach because of couple of things:

1 - resourcesDB for example have only 2 collections,

    Logs      ---> logging every resources change (building built, war lost etc.) 
    Resources ---> 1 document per user holding how much he have.

meaning that this DB will not hold any huge data (logging will be deleted after some time), so setting an instance just for those 2 collections is looking like a huge over kill, even if this instance will not have its own server and will share server with the service.

2 - maintaining and doing backups will require some extra work that I'm not sure that its needed in my case.

Each of the service will update his DB quite often, resource is doing calculation every second to see how much each user have, so it will update the db 'users amount' per second, aside with the rest of the requests like build a building (reduce resources) and other updates, Chat service will update the DB every private message or chat message, Battle service will update troops place and stats and more stuff every second for each battle that is going on.

I'm thinking to chance that structure to one of the next:

  1. maybe have only 1 instance of database with separated DB, so each service will connect to a separated database, but i read that it will still be a hassle to maintain backups of the DB's and scaling.

  2. have 1 database, and have all the collections in it, every collection will be accessed by a different service and will never be accessed by 2, so i have nothing to worry about 2 service changing same collection.

  3. leave it as it is, and go with many instances of mongoDB

few questions about the methods above:

  1. what should i choose? having 1 database sounds good as it will save me a lot of resources and will make my life easier, but separating the services to many database sound logic to me as one service don't need to see collections or be able to access a collection that have nothing to do with his job.

  2. I'm not sure how mongoDB handle many connections to the same instance, or to same database, as you can see it might get a lot of calls in a second even if i have pretty low amount of users.

  3. I read about mongo Sharding, from what i understood it happens per database and not on the instance level, that's why i'm thinking about the 1 database option, as it quite useful in my case, am i right?

  • No one can help with this? no insights? – Yan Mayatskiy Dec 1 '16 at 17:24
3

It's sensible to think about how to scale your application in future, but you can start with a simpler deployment that can grow with your requirements.

what should i choose? having 1 database sounds good as it will save me a lot of resources and will make my life easier, but separating the services to many database sound logic to me as one service don't need to see collections or be able to access a collection that have nothing to do with his job.

My recommendation would be to set up a single MongoDB deployment (ideally starting with a replica set or hosted service with high availability & data redundancy):

  • A single deployment of MongoDB can have multiple databases; a MongoDB deployment per service adds unnecessary admin complexity.

  • For your use case it sounds reasonable to have a database per service and setup a user per service so actions are limited to direct interaction with the expected database.

  • A database per service also limits potential impact of administrative actions (eg. a foreground index build) that may create database-level contention.

Later you can scale/tune the deployment starting with options like directoryPerDB (eg. faster storage for some databases) or partitioning your data using sharding.

I'm not sure how mongoDB handle many connections to the same instance, or to same database, as you can see it might get a lot of calls in a second even if i have pretty low amount of users.

This is a more nuanced question as it depends on server resources as well as what those connections are doing and how/if your driver implements connection pooling. Ideally I would load test your actual application and determine how resource usage scales with increased user activity. If you are administering your own MongoDB deployment you should review the production notes and in particular make sure you have your ulimit settings increased appropriately to ensure you don't run out of file descriptors.

If you're concerned about ease of scaling you could also consider a hosted database-as-a-service solution (for example, MongoDB Atlas).

I read about mongo Sharding, from what i understood it happens per database and not on the instance level, that's why i'm thinking about the 1 database option, as it quite useful in my case, am i right?

Sharding is initially enabled at the database level but the partitioning happens at the collection level based on a shard key. You can have a mix of sharded/unsharded databases and sharded/unsharded collections in the same MongoDB deployment.

  • Thank you thank you and another thank you, i'm currently working on my localhost so i have no point in testing the things you mentioned now, but i will sure come back to this question when i'll go live. – Yan Mayatskiy Dec 2 '16 at 10:42
  • @YanMayatskiy Although it's probably still early in your application development cycle, I would start to set up some of the expected data structures and access control (eg. database-per-service and appropriate per-service users/roles) in order to proactively resolve some concerns. For example, if your services are configured with distinct user accounts and properly restricted from the outset you can avoid lazy or accidental cross-talk between services and will have configuration placeholders ready for future non-local deployments. – Stennie Dec 3 '16 at 12:40
  • @YanMayatskiy You can defer tuning & operational deployment (eg. setting up a replica set) until later in your development cycle, but it would still be worth considering how your application handles network errors and other exceptions. I would highly recommended reading the blog post: How To Write Resilient MongoDB Applications, which was written by one of the authors of PyMongo (the official MongoDB Python driver). – Stennie Dec 3 '16 at 12:48
  • Thanks for the tips, i'm currently a single developer for that project, so my main focus is for making the actual game, and setting things up to be changeable very easily in the future, roles per service is not on my list for now, the blog post helped a lot. This is great answer to get back to. – Yan Mayatskiy Dec 3 '16 at 20:38

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.