4

I have a MongoDB in my Development environment running on Windows, the server has 64GB of RAM, but MongoDB has only consumed 26.9GB (here we are building the database), so it should be able to hold all the indexes and data in ram:

DB Stats:
Data Size: 27.8071 GB
Index Size: 3.96 GB

After 3 or 4 hours running, insert performance drops to only 2-3k inserts per second, despite all indexes being in RAM (or they should be).

However, I've noticed if I restart the server, insert speeds rates increase to x3 or x4.

I'm wondering why this is, and could it be caused by running on Windows?

The collections being inserted (with any frequency) have (at this time) 20 mil, 44 mil, and 51 mil.

The first collection of 20mil has a very random index based on a 256 bit hash, the other two collections are indexed on ObjectId. This is the bottle neck I cannot shift without sharding, but it is concerning that the insert rate changes so much after a restart.

I do not want to periodically restart my Primary node, causing clients to have to fail over to a secondary.

Edit: I should also say, I am running with write concern unacknowledged.

Please find example image of logs, apologies I did not capture a text output of this log. enter image description here

Some typical slow queries when they are popped up by the profiler, set at 100ms:

2015-08-18T11:56:34.768+0100 I COMMAND  [conn7] command Slice_BTC.$cmd command: insert { insert: "Transactions", writeConcern: { w: 0 }, ordered: false, documents: 765 } keyUpdates:0 writeConflicts:0 numYields:
 acquireCount: { w: 7 } }, Database: { acquireCount: { w: 7 } }, Collection: { acquireCount: { w: 7 } } } 790ms
2015-08-18T11:56:35.184+0100 I QUERY    [conn10] query Slice_BTC.Outputs query: { $query: { t: ObjectId('55d2cf76137e6e233c231ecf'), i: 96 } } planSummary: IXSCAN { t: 1, i: 1 } ntoreturn:1 ntoskip:0 nscanned:1
:0 writeConflicts:0 numYields:0 nreturned:1 reslen:224 locks:{ Global: { acquireCount: { r: 1 } }, Database: { acquireCount: { r: 1 } }, Collection: { acquireCount: { r: 1 } } } 378ms

Typical size of Transactions object: 178 bytes. Typical size of Outputs object: 204 bytes.

Improve this question
9
  • You mention "master node" here. How many other members are in the set? Are they all of the same configuration? Is there a noted "lag" on any of those servers where replication is falling behind?
    – Blakes Seven
    Aug 18, 2015 at 10:21
  • In this instance there are no other members as it is development, running on a dev machine. In production there are 3 members in the replica set. Primary was in intended meaning.
    – James
    Aug 18, 2015 at 10:27
  • Assuming that this is mosly inserts in load. The query and update counts are quite level where getmore is basically non-existant. So guessing there are upsert operations in here as well. It would be good to narrow down if specific operations are getting slower and also if it is specific questions. Your initial concern is generally warranted since like a lot of projects, MongoDB is developed for a *nix platform first with of course the majority of testing being there. You do seem write heavy, so is their journaling on if no write concern response? Also which engine? Still MMAP?
    – Blakes Seven
    Aug 18, 2015 at 10:43
  • Thanks for your support @BlakesSeven. There are no upsert operations (well there are 2, on different collections with only 300k records). Journaling is disabled as this is a DB build and so can be re-run if bad. I am using WiredTiger currently as the DB is in the cloud and high IOPS disk space is expensive, final DB size on disk is approx. 200 GB. I do not have a dedicated Linux machine with the IOPS necessary to build this DB without hosting it in a VM, which would not be a fair comparison. Server is an i7 with 6 Cores so compressions should not be an issue. CPU on mongod is pegged at 14%.
    – James
    Aug 18, 2015 at 10:50
  • 2
    All of this "additional information" is better served by edits to your question rather than posting additional comments and making people read through them. Sure memory might be capping out at a certain point, but that may be just a consequence of the data that is in memory is already sufficient to support the operations that are happening. It would be more interesting if you could watch the process and edit your question with additional information such as if page swapping is happening or other interesting stats. More information in the question, and less comments so it is easy to read.
    – Blakes Seven
    Aug 18, 2015 at 11:50

1 Answer 1

Reset to default
0

I believe the slowdown is caused by running in Windows. I am going to attempt this on a Linux box once our new server arrives.

However, the 26GB limit is because of the WiredTiger default configuration which only caches 50% up to max ram.

You can change this using the http://docs.mongodb.org/v3.0/reference/configuration-options/#storage.wiredTiger.engineConfig.cacheSizeGB option.

Improve this answer
2
  • 3
    Any results regarding your assumption that it might be a Windows issue?
    – John K. N.
    Sep 1, 2017 at 9:26
  • 1
    We see performance improvement after restart of the cluster on Ubuntu as well. This may not be a windows issue?
    – Victor Pudeyev
    Sep 6, 2017 at 21:25

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge that you have read and understand our privacy policy and code of conduct.

Not the answer you're looking for? Browse other questions tagged or ask your own question.