I recently started running into the Sort exceeded memory limit of... exception on an aggregation pipeline in MongoDB. I got around this by setting AllowDiskUse=True for the aggregation. However, I read that this can significantly slow down performance.
I would like to optimize the query but I cannot figure out how to measure how much memory is being used by the aggregation. I also need to run the pipeline on multiple collections in production so I need to know how the memory usage will scale with collection-size.
Does anyone know how to do this? The .explain() method does not seem to show any information on memory usage and I haven't found anything in the docs/forums.
