0

I have a newly installed MongoDB server running on an AWS Ubuntu EC2. The server has nothing else installed on it. The DB currently contains 35 documents of 3 MB each, meaning less than 110 MB of data.

I'm running an aggregation query that contains 2 stages: a simple $match stage that makes sure that only 31 documents are aggregated, and a $group stage with $accumulator inside (that stage doesn't allocate much new space, and even if it did, I expect that when handling with 3 MB documents it won't be much more). The query's result should be a single 3 MB documents that is a merger of all of the above.

Running the query when only 3 documents exist - finishes without a problem. But with 35 - I consistently get the following error:

MongoError: Out of memory
    at MessageStream.messageHandler (/app/node_modules/mongodb/lib/cmap/connection.js:268:20)
    at MessageStream.emit (events.js:314:20)
    at processIncomingData (/app/node_modules/mongodb/lib/cmap/message_stream.js:144:12)
    at MessageStream._write (/app/node_modules/mongodb/lib/cmap/message_stream.js:42:5)
    at writeOrBuffer (_stream_writable.js:352:12)
    at MessageStream.Writable.write (_stream_writable.js:303:10)
    at Socket.ondata (_stream_readable.js:717:22)
    at Socket.emit (events.js:314:20)
    at addChunk (_stream_readable.js:307:12)
    at readableAddChunk (_stream_readable.js:282:9)

/etc/mongod.conf:

# mongod.conf

# for documentation of all options, see:
#   http://docs.mongodb.org/manual/reference/configuration-options/

# Where and how to store data.
storage:
  dbPath: /var/lib/mongodb
  journal:
    enabled: true
#  engine:
#  mmapv1:
#  wiredTiger:

# where to write logging data.
systemLog:
  destination: file
  logAppend: true
  path: /var/log/mongodb/mongod.log

# network interfaces
net:
  port: 27017
  bindIp: 127.0.0.1,<IP>



# how the process runs
processManagement:
  timeZoneInfo: /usr/share/zoneinfo

#security:
security:
    authorization: "enabled"

#operationProfiling:

#replication:

#sharding:

## Enterprise-Only Options:

#auditLog:

#snmp:

I've read a lot online and couldn't solve it, I've tried:

  1. Upgrading the server's hardware to t2.large (8 GB RAM).
  2. Adding allowDiskUse to the query:
const options = {
  allowDiskUse: true,
  // explain: true,
}
mongoConn.db(dbConfig.database).collection(collection).aggregate(aggregationPipeline, options).toArray((err: MongoError, result: any) => {
  1. With htop, I can see the total memory usage of 210M/7.68G after restarting MongoDB, and during the query it climbs to a peak of 691M/7.68G, fails, and remains on 627M/7.68G afterward. enter image description here

  2. With db.enableFreeMonitoring() I can see a constant 2 GB virtual memory usage, with peaks to 2.1 GB: image

  3. The result of db.serverStatus().tcmalloc.tcmalloc.formattedString: enter image description here

  4. The result of db.serverStatus().mem: enter image description here

Summary: I know that MongoDB has a 100MB memory limit, but I guess that it shouldn't reach it with 3 MB documents, and allowDiskUse. What am I missing here?

  • Changing the hardware to t3.xlarge (16GB RAM) didn't solve it as well – A-S Oct 4 at 11:57
  • 1
    How does your aggregationPipeline look like – Wernfried Domscheit Oct 4 at 12:59
  • That error looks more like a problem in the client. Does the system user running the query have a different set of ulimits? – Joe Oct 9 at 5:12

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Browse other questions tagged or ask your own question.