0

I work on a mongodb sharded cluster (3.0.2 i know it's very outdated, not my call), on which i have a sharded db and a sharded collection on which i have multiple indexes.

One of the indexes is a TTL index (24h only, 86400) on a date field named d, used to expire old data, this leads to almost fixing the number of entries in the collection within our use case. Given the fixed structure and model of the data, this means an almost fixed size of the collection and it's indexes, it has always been the case for index and collection sizes throughout the whole cluster.

Our issue is, we have another index on this collection, on a field named k which is a large string.

ex : GigabitEthernet_XXXX-XXXXX-01_ABC_DEFG-HIJKLMNOP-5-QRS_TUV_7_0/0/1

This index keeps growing indefinitely in size, and causes memory outages and we don't know why. Any help ?

Knowing that the TTL index always expires/removes data from the collection, isn't the second index, on the k field supposed to also remove the references to that data, hence be fixed or at least not be evergrowing in size ?

We use this same technique (TTL index + other indexes) on all our collections and it always works in terms of fixing the sizes, not on this collection though. It is also important to note that other collections have a much smaller k field than this one, is that the issue ? Any explanations ? Any mongodb docs about this ?

Happy to provide any needed details to explain the issue.

Thanks in advance

0

Compact the collection in your staging environment. If index size decreases, space is taken up by overhead.

Compare the size of compacted collection & index to the size of both pre compaction, identify the overhead amount/percentage.

1
  • I am afraid that wouldn't help. We use the same the same technique (TTL index + the other indexes) on multiple other collections, of the same type, and they don't present this issue. The only difference is the "k" field in this collection is much larger. Then why would the overhead keep increasing, knowing that the data is fixed in number of entries per 24h ? I even tried dropping and recreating the collection with it's indexes, behaviour is still the same – MrRobot Aug 5 '20 at 13:50

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.