2

My current storage engine is WiredTiger and its compression level is as default, snappy. I've come across MongoDB documentation and it's been mentioned that using zlib compress better but needs more CPU.

I want to know will zlib store more data in memory compared to snappy as it compress the data? I have a server with 16 CPU cores. As RAM is more expensive I'd rather to save on memory in case it keeps more data.

Is this correct? Can I blindly switch to zlib to cache more data and improve read performance?

NOTE: Our server is read intensive.

2

The documentation at https://docs.mongodb.com/manual/core/wiredtiger/#memory-use says that collection data is only compressed on disk, so uncompressed in memory. Indexes keep their prefix compression in memory, but not the block-level compression.

So changing compression algorithms is not going to directly reduce the amount of memory needed to keep a read-heavy workload efficient (i.e. to make sure its working set stays in RAM) unless you are already memory-starved in which case better compression may help by reducing IO as the storage system is thrashed because any data for the DB in the OS's buffers & cache will be compressed. The only way to tell for sure is to benchmark a realistic workload on production-like data in a test environment with each combination of options you are considering.

The "unless you are already memory-starved" is significant here though: once you are in that state the best the compression is likely to do is improve your performance from very very slow to just very slow.

One exception to the above would be any query that needs to read a data-set that is too large to fit into any practical amount of memory, in which case you may see significant improvements: all the data the query needs will need to be read from the IO subsystem anyway, and the compression is likely to help this. We'd need to know a lot more about your application's data and workloads to give specific hints as to whether this would have a noticeable effect in your case, and even then the only way to be sure is, again, to run benchmarks.

NOTE: this is not the case for all databases. For example with MS SQL Server's compression options the data in pages in RAM are compressed just as they are on disk. This reduces RAM use by the buffer pool at the expense of CPU time on every read of each page. When data is uncompressed in RAM the decompression CPU hit is only experienced as the data is loaded from disk so won't affect subsequent reads until that page/block/document is evicted because it hasn't been referred to recently.

Can I blindly...

The short answer is: No.

A longer answer is: Nnnnnoooooooooooooooooooooooooooooooooooooooooooooooooo...

Less facetiously: blindly doing anything on a production database and/or application is dangerous so it is never a recommended course of action. Never do this if you value your users and your own sanity. Always test in a dev/test environment first no matter how much you trust any source of advice that suggests otherwise. It may feel like wasting time when you test and find no ill effects, but you will at some point in your career be very glad of having applied due diligence and saved yourself from a nasty and embarrassing event on a application in production! If you don't follow due diligence in this way, you may experience unplanned CV update events.

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.