MongoDB crashed twice in 6 hours today due to invariant() failure. We have been using MongoDB 3.6.3MongoDB 3.6.3
for more than a year now, but this is the first time that we saw this issue. The extract from the log files is given below for reference -
Log from 1st failure:
2019-02-27T08:44:51.411+0530 E STORAGE [conn3385299] WiredTiger error (12) [1551237291:411637][2344:2000827264], file:collection-17055--5635302636308412295.wt, WT_CURSOR.search: C:/MREYE/MongoDB/data\collection-17055--5635302636308412295.wt: handle-read: ReadFile: failed to read 4096 bytes at offset 720363520: Not enough storage is available to process this command.
: Not enough space 2019-02-27T08:44:51.412+0530 F - [conn3385299] Invariant failure: seekRet resulted in status UnknownError: 12: Not enough space at src\mongo\db\storage\wiredtiger\wiredtiger_record_store.cpp 1734 2019-02-27T08:44:51.412+0530 F - [conn3385299]
***aborting after invariant() failure
2019-02-27T08:44:51.411+0530 E STORAGE [conn3385299] WiredTiger error (12) [1551237291:411637][2344:2000827264], file:collection-17055--5635302636308412295.wt, WT_CURSOR.search: C:/MREYE/MongoDB/data\collection-17055--5635302636308412295.wt: handle-read: ReadFile: failed to read 4096 bytes at offset 720363520: Not enough storage is available to process this command.
: Not enough space 2019-02-27T08:44:51.412+0530 F -
[conn3385299] Invariant failure: seekRet resulted in status UnknownError: 12: Not enough space at src\mongo\db\storage\wiredtiger\wiredtiger_record_store.cpp 17342019-02-27T08:44:51.412+0530 F - [conn3385299]
***aborting after invariant() failure
Log from 2nd failure:
2019-02-27T13:44:11.594+0530 E STORAGE [conn20] WiredTiger error (12) [1551255251:594637][54524:2000827264], file:collection-17071--5635302636308412295.wt, WT_CURSOR.search: memory allocation of 28696 bytes failed: Not enough space
2019-02-27T13:44:11.594+0530 F - [conn20] Invariant failure: seekRet resulted in status UnknownError: 12: Not enough space at src\mongo\db\storage\wiredtiger\wiredtiger_record_store.cpp 1734
2019-02-27T13:44:11.595+0530 F - [conn20]
***aborting after invariant() failure
2019-02-27T13:44:11.594+0530 E STORAGE [conn20] WiredTiger error (12) [1551255251:594637][54524:2000827264], file:collection-17071--5635302636308412295.wt, WT_CURSOR.search: memory allocation of 28696 bytes failed: Not enough space
2019-02-27T13:44:11.594+0530 F - [conn20] Invariant failure: seekRet resulted in status UnknownError: 12: Not enough space at src\mongo\db\storage\wiredtiger\wiredtiger_record_store.cpp 1734
2019-02-27T13:44:11.595+0530 F - [conn20]
***aborting after invariant() failure
Note:
The MongoDB is on C drive and there is sufficient space left (276 GB) on that drive. The MongoDB data folder size itself is only 65 GB. Any idea what might have caused this issue?