I'm currently investigating the possibility of using mongodb. This would contain the a collection of 100million+ documents. I need a process that will be constantly running deleting documents older than some expiry data. I will have new documents constantly going into this. I will also have big updates occuring that may need to update X million document to change the expiry date because a tenant has changed their retention policy. At the same time a front end will be reading this data to show a report which can be filtered.
Reading the mongodb documentation it looks like when I do a write I'll be locking out the reads until the writes have completed. Is there a better way to prevent the expiry of old documents and updating of expiry dates blocking the front ended display this data in a report?