0

We have a MongoDb instance which contains two main db use about 30G of disk.

>show dbs;

admin             (empty)
l_comment        17.945GB
l_recommend      15.946GB
lifeix_relation   1.953GB
local             2.077GB
test              (empty)

Some time the mongostat command showed that the mongo use very much memory up to 61G in about 1 or 2 minutes (memory of the machine is 64G).

Below is some output of mongostat, it shows that there is no index miss during the time, but the date of column 'res' is growing fast, and res size is much bigger than db size showed use show dbs .

So what can cause this?

Thanks for any tips and help.

    *0     29      1     *0       0     7|0       0  35.9g  72.3g  1.67g      0       l_comment:0.0%          0       0|0     0|0     4k    36k   121  PRI   12:24:58 
    *0     23     *0     *0       0     8|0       0  35.9g  72.3g  1.68g      0       l_comment:0.0%          0       0|0     0|0     4k    39k   121  PRI   12:24:59 
    *0     18     *0     *0       0     8|0       0  35.9g  72.3g  1.68g      0       l_comment:0.0%          0       0|0     0|0     3k    41k   121  PRI   12:25:00 
    *0     22     *0     *0       0     5|0       0  35.9g  72.3g  1.68g      1       l_comment:0.0%          0       0|0     0|0     3k    22k   121  PRI   12:25:01 
     1     21      4     *0       4    12|0       0  35.9g  72.3g  1.68g      0       l_comment:0.2%          0       0|0     0|0     4k    44k   121  PRI   12:25:02 
     1     15      4     *0       3     9|0       0  35.9g  72.3g  1.67g      0       l_comment:0.3%          0       0|0     0|0     3k     6k   121  PRI   12:25:03 
    *0     11     *0     *0       0     3|0       0  35.9g  72.3g  1.68g      0       l_comment:0.0%          0       0|0     0|0     1k    19k   121  PRI   12:25:04 
insert  query update delete getmore command flushes mapped  vsize    res faults            locked db idx miss %     qr|qw   ar|aw  netIn netOut  conn repl       time 
     1     30      5     *0       3    15|0       0  35.9g  72.5g   1.8g      0       l_comment:0.1%          0       0|0     1|0     5k    24k   121  PRI   12:25:05 
    *0     16      1     *0       1     6|0       0  35.9g  73.1g  2.45g      0       l_comment:0.0%          0       0|0     3|0     2k    15k   121  PRI   12:25:06 
     2     29      7     *0       6    17|0       0  35.9g  74.2g  3.56g      0       l_comment:0.5%          0       0|0     3|0     6k    38k   121  PRI   12:25:07 
    *0     13      1     *0       0     6|0       0  35.9g  74.9g  4.53g      0       l_comment:0.0%          0       0|0     6|0     1k    48k   121  PRI   12:25:08 
     2     27      6     *0       5    17|0       0  35.9g  75.5g  5.47g      0       l_comment:2.2%          0       0|0     9|0     5k    21k   121  PRI   12:25:09 
     1     25      6     *0       5    16|0       0  35.9g  75.7g  5.88g      1       l_comment:2.5%          0       0|0    13|0     5k    15k   121  PRI   12:25:10 
     2     39      7     *0       8    20|0       0  35.9g  77.4g  7.94g      0       l_comment:1.5%          0       0|0    14|0     7k    54k   121  PRI   12:25:11 
    *0     14      1     *0       0     6|0       0  35.9g  78.3g  7.86g      0       l_comment:4.8%          0       0|0    17|0     2k    37k   121  PRI   12:25:12 
    *0     15     *0     *0       0     8|0       0  35.9g  78.9g  9.51g      0       l_comment:0.0%          0      19|1    21|0     2k    29k   121  PRI   12:25:13 
     2     49      7     *0       8    23|0       0  35.9g  79.4g  10.7g      4       l_comment:0.2%          0       0|0    22|0     9k    46k   121  PRI   12:25:14 
insert  query update delete getmore command flushes mapped  vsize    res faults            locked db idx miss %     qr|qw   ar|aw  netIn netOut  conn repl       time 
    *0     21     *0     *0       0    10|0       0  35.9g  80.7g  11.2g      0       l_comment:5.8%          0       0|0    25|0     3k    36k   121  PRI   12:25:15 
     1     28      3     *0       4    13|0       0  35.9g  80.9g  11.7g      0       l_comment:0.2%          0       0|0    27|0     5k    44k   121  PRI   12:25:16 
     1     23      4     *0       4    16|0       0  35.9g    81g  11.9g      1       l_comment:0.5%          0       0|0    31|0     4k    30k   121  PRI   12:25:17 
    *0     19      1     *0       0     6|0       0  35.9g  81.5g  12.3g      8       l_comment:0.0%          0      37|1    38|0     2k    31k   121  PRI   12:25:18 
    *0     34     *0     *0       0    13|0       0  35.9g    82g  13.5g      2       l_comment:2.2%          0      31|0    35|0     6k    54k   121  PRI   12:25:19 
    *0     13     *0     *0       0     7|0       1  35.9g  82.9g  14.9g      0       l_comment:0.0%          0      36|0    38|0     2k    17k   121  PRI   12:25:20 
    *0     20     *0     *0       0    10|0       0  35.9g  84.5g  16.7g      0       l_comment:0.0%          0      33|0    39|0     3k    29k   121  PRI   12:25:21 
    *0     20     *0     *0       1     6|0       0  35.9g  85.6g  17.8g      1       l_comment:0.0%          0      31|0    40|0     3k    20k   121  PRI   12:25:22 
    *0     19      1     *0       1     5|0       0  35.9g  86.9g  19.4g      1       l_comment:0.0%          0       0|0    43|0     2k    16k   121  PRI   12:25:23 
     1     25      4     *0       5    14|0       0  35.9g  87.3g  19.9g      0       l_comment:0.2%          0       0|0    43|0     5k    23k   121  PRI   12:25:24 
insert  query update delete getmore command flushes mapped  vsize    res faults            locked db idx miss %     qr|qw   ar|aw  netIn netOut  conn repl       time 
    *0     28      3     *0       1    12|0       0  35.9g  88.1g  20.9g      1       l_comment:0.0%          0      43|1    44|0     5k    25k   121  PRI   12:25:25 
    *0     26     *0     *0       0     9|0       0  35.9g  89.1g    22g      0       l_comment:0.0%          0       0|0    46|0     4k    22k   121  PRI   12:25:26 
    *0     14     *0     *0       0     6|0       0  35.9g  89.1g  20.6g      1       l_comment:5.8%          0       0|0    48|0     2k    27k   121  PRI   12:25:27 
    *0     32     *0     *0       0    15|0       0  35.9g  89.1g    21g      3       l_comment:0.0%          0      68|1     6|0     3k    20k   121  PRI   12:25:28 
     1     35      4     *0       3    17|0       0  35.9g  89.8g  22.2g      2       l_comment:0.2%          0       0|0    58|0     8k    71k   121  PRI   12:25:29 
    *0     24     *0     *0       0    12|0       0  35.9g  89.8g  21.6g      0       l_comment:4.0%          0       0|0    64|0     4k    22k   121  PRI   12:25:30 
    *0     29     *0     *0       0    11|0       0  35.9g  89.9g  22.1g      0       l_comment:0.0%          0      74|1    75|0     4k    37k   121  PRI   12:25:32 
     1     40      4     *0       4    17|0       0  35.9g  90.7g  23.2g     14       l_comment:0.2%          0      25|0    48|0     8k    56k   121  PRI   12:25:33 
    *0     24     *0     *0       0    12|0       0  35.9g  91.8g  24.3g     20       l_comment:1.4%          0      41|0    73|0     3k    21k   121  PRI   12:25:34 
     1     23      4     *0       4    11|0       0  35.9g    92g  23.9g      6       l_comment:3.5%          0      47|0    68|0     4k    20k   121  PRI   12:25:35 
insert  query update delete getmore command flushes mapped  vsize    res faults            locked db idx miss %     qr|qw   ar|aw  netIn netOut  conn repl       time 
    *0     19     *0     *0       0    10|0       0  35.9g  93.2g  25.5g      0       l_comment:0.0%          0      54|0    80|0     3k    24k   121  PRI   12:25:36 
    *0     17     *0     *0       0     5|0       0  35.9g  94.6g  26.1g      0       l_comment:3.7%          0      88|0    88|0     1k     9k   121  PRI   12:25:37 
    *0     31     *0     *0       0     9|0       0  35.9g  95.4g  27.4g      3       l_comment:0.0%          0      32|0    83|0     5k    52k   121  PRI   12:25:38 
    *0     35      3     *0       1    12|0       0  35.9g  96.4g    29g      1       l_comment:0.0%          0      29|0    85|0     6k    56k   121  PRI   12:25:39 
    *0     15     *0     *0       0     9|0       0  35.9g  97.6g  30.6g      0       l_comment:0.2%          0      53|0    87|0     2k    12k   121  PRI   12:25:40 
    *0     32     *0     *0       0    11|0       0  35.9g  98.7g  32.1g      8       l_comment:0.0%          0      61|0    93|0     5k    56k   121  PRI   12:25:41 
     1     33      4     *0       4     9|0       0  35.9g  99.9g  33.3g     19       l_comment:0.1%          0      55|0    90|0     6k    26k   121  PRI   12:25:42 
    *0     12     *0     *0       0     4|0       0  35.9g   100g  31.9g      1       l_comment:0.0%          0     104|0   104|0   841b    12k   121  PRI   12:25:43 
    *0     20      5     *0       2    12|0       0  35.9g   101g  32.8g      0       l_comment:6.9%          0      98|1   100|0     5k    23k   121  PRI   12:25:44 
    *0     18     *0     *0       0     4|0       0  35.9g   101g  33.2g      1       l_comment:0.1%          0      93|1    99|0     3k    39k   121  PRI   12:25:46 
insert  query update delete getmore command flushes mapped  vsize    res faults            locked db idx miss %     qr|qw   ar|aw  netIn netOut  conn repl       time 
    *0      2     *0     *0       0     2|0       0  35.9g   101g  33.6g      0       l_comment:0.0%          0      69|0    93|0   490b    13k   121  PRI   12:25:47 
    *0      9     *0     *0       0     1|0       0  35.9g   101g  34.3g      0       l_comment:0.1%          0      66|0    95|0   966b     4k   121  PRI   12:25:48 
    *0      8     *0     *0       1     1|0       0  35.9g   101g  34.7g      4       l_comment:0.0%          0       0|0   100|0   932b     6k   121  PRI   12:25:49 
    *0      7     *0     *0       0     2|0       0  35.9g   102g    36g      0       l_comment:0.0%          0       0|0   100|0   724b     3k   121  PRI   12:25:50 
    *0      5     *0     *0       0     1|0       0  35.9g   104g  37.2g      1       l_comment:0.0%          0      99|1   101|0   556b     4k   121  PRI   12:25:52 
    *0     10     *0     *0       0     1|0       0  35.9g   104g  37.5g      0       l_comment:0.1%          0      79|0    91|0     1k     4k   121  PRI   12:25:53 
    *0      2     *0     *0       0     1|0       0  35.9g   104g  37.6g      0       l_comment:0.0%          0      45|0    69|0   267b     4k   121  PRI   12:25:54 
    *0      5     *0     *0       1     1|0       0  35.9g   105g    39g      2       l_comment:0.0%          0      15|0    86|0   600b     5k   121  PRI   12:25:55 
    *0      3      1     *0       1     1|0       0  35.9g   107g  40.6g      0       l_comment:0.0%          0       0|0   100|1   395b     4k   121  PRI   12:25:56 
    *0      8     *0     *0       0     1|0       0  35.9g   108g  41.8g      3       l_comment:0.0%          0     100|1   101|0   845b    39k   121  PRI   12:25:57 
insert  query update delete getmore command flushes mapped  vsize    res faults            locked db idx miss %     qr|qw   ar|aw  netIn netOut  conn repl       time 
    *0      4     *0     *0       0     1|0       0  35.9g   108g  42.6g      0       l_comment:0.1%          0       0|0   100|0   472b     3k   121  PRI   12:25:58 
    *0      7     *0     *0       0     1|0       0  35.9g   108g  42.7g      2       l_comment:0.0%          0       0|0   100|0   821b     7k   121  PRI   12:25:59 
    *0      5     *0     *0       0     1|0       0  35.9g   109g  43.4g      0       l_comment:0.0%          0     101|1   104|0   304b     4k   121  PRI   12:26:01 
    *0      6     *0     *0       0     1|0       0  35.9g   109g  43.4g      1       l_comment:0.0%          0      97|1   100|0   966b     5k   121  PRI   12:26:02 
    *0      3     *0     *0       1     1|0       0  35.9g   110g  44.2g      1       l_comment:0.0%          0      97|1   100|0   438b     3k   121  PRI   12:26:03 
    *0      7     *0     *0       0     1|0       0  35.9g   110g  44.7g      2       l_comment:0.0%          0       0|0   100|0   724b     6k   121  PRI   12:26:04 
    *0      7      2     *0       1     4|0       0  35.9g   111g  44.8g      0       l_comment:0.0%          0       0|0   100|0     1k     4k   121  PRI   12:26:05 
    *0      5      1     *0       0     1|0       0  35.9g   111g  45.3g      2       l_comment:0.0%          0     103|1   104|0   304b     4k   121  PRI   12:26:06 
    *0      8      2     *0       1     3|0       0  35.9g   112g    46g      4       l_comment:0.1%          0      98|1   100|0     1k    80k   121  PRI   12:26:08 
    *0      5      1     *0       0     1|0       0  35.9g   112g  46.6g      0       l_comment:0.0%          0      66|0    88|0   593b     4k   121  PRI   12:26:09 
insert  query update delete getmore command flushes mapped  vsize    res faults            locked db idx miss %     qr|qw   ar|aw  netIn netOut  conn repl       time 
    *0      4     *0     *0       0     1|0       0  35.9g   112g  46.6g      0       l_comment:0.0%          0       0|0   100|0   472b     3k   121  PRI   12:26:10 
    *0     *0     *0     *0       0     1|0       0  35.9g   112g  46.7g      0       l_comment:0.0%          0      99|1   100|0    62b     3k   121  PRI   12:26:12 
    *0      9      3     *0       2     3|0       0  35.9g   112g  46.9g      1       l_comment:0.0%          0      52|0    68|0     1k     5k   121  PRI   12:26:13 
    *0      2     *0     *0       0     1|0       0  35.9g   112g    47g      0       l_comment:0.1%          0      71|0    88|0   267b     3k   121  PRI   12:26:14 
    *0     *0     *0     *0       0     1|0       0  35.9g   112g  47.1g      0       l_comment:0.0%          0     100|1   101|0    62b     3k   121  PRI   12:26:15 
    *0      4     *0     *0       0     1|0       0  35.9g   113g  47.3g      0       l_comment:0.0%          0      25|0    69|0   472b     3k   121  PRI   12:26:16 
    *0      5     *0     *0       0     1|0       0  35.9g   114g  48.3g      0       l_comment:0.0%          0      98|1   100|0   556b     3k   121  PRI   12:26:17 
    *0      4     *0     *0       0     1|0       0  35.9g   114g    49g      2       l_comment:0.0%          0      76|0    92|0   472b     8k   121  PRI   12:26:18 
    *0      3     *0     *0       1     1|0       0  35.9g   115g  49.9g      0       l_comment:0.0%          0      67|0    87|0   438b     3k   121  PRI   12:26:19 
    *0      3     *0     *0       0     2|0       1  35.9g   115g  49.9g      0       l_comment:0.1%          0       0|0   100|0   351b     3k   121  PRI   12:26:20 
insert  query update delete getmore command flushes mapped  vsize    res faults            locked db idx miss %     qr|qw   ar|aw  netIn netOut  conn repl       time 
    *0     *0     *0     *0       0     1|0       0  35.9g   115g  49.9g      0       l_comment:0.0%          0      95|1   100|0    62b     3k   121  PRI   12:26:22 
    *0     16     *0     *0       0     1|0       0  35.9g   115g  49.9g      0       l_comment:0.0%          0     101|1   102|0   966b     4k   121  PRI   12:26:23 
    *0      2      2     *0       1     2|0       0  35.9g   115g  50.1g      2       l_comment:0.0%          0      97|1   100|0   883b     5k   121  PRI   12:26:25 
    *0      3     *0     *0       0     1|0       0  35.9g   115g  50.1g      3       l_comment:0.1%          0      95|1   100|0   388b     4k   121  PRI   12:26:27 
    *0      8     *0     *0       0     1|0       0  35.9g   115g  50.2g      1       l_comment:0.0%          0      96|1   101|0   761b     4k   121  PRI   12:26:29 
    *0      3     *0     *0       0     1|0       0  35.9g   115g  50.2g      1       l_comment:0.0%          0      99|1   100|0   472b    18k   121  PRI   12:26:31 
    *0      5     *0     *0       1     1|0       0  35.9g   115g  50.3g      0       l_comment:0.0%          0      92|1   100|0   643b     4k   121  PRI   12:26:33 
    *0      9     *0     *0       0     1|0       0  35.9g   115g  50.3g      0       l_comment:0.0%          0      61|0    83|0     1k     5k   121  PRI   12:26:34 
    *0      5      2     *0       1     3|0       0  35.9g   115g  50.3g      0       l_comment:0.0%          0       0|0   100|0     1k     4k   121  PRI   12:26:35 
    *0      2      1     *0       0     2|0       0  35.9g   115g  50.3g      0       l_comment:0.0%          0      99|1   102|0   183b     3k   121  PRI   12:26:37 
insert  query update delete getmore command flushes mapped  vsize    res faults            locked db idx miss %     qr|qw   ar|aw  netIn netOut  conn repl       time 
    *0      3      1     *0       1     1|0       0  35.9g   115g  50.4g      1       l_comment:0.0%          0      96|1   100|0   559b     4k   121  PRI   12:26:39 
    *0      5     *0     *0       0     1|0       0  35.9g   115g  50.4g      3       l_comment:0.0%          0      98|1   100|0   550b     4k   121  PRI   12:26:41 
    *0      4     *0     *0       0     1|0       0  35.9g   115g  50.4g      0       l_comment:0.0%          0      98|1   100|0   472b     5k   121  PRI   12:26:42 
    *0     16      3     *0       2     3|0       0  35.9g   115g  50.4g      1       l_comment:0.0%          0      95|1   100|0     2k     5k   121  PRI   12:26:45 
    *0      8     *0     *0       0     1|0       0  35.9g   115g  50.5g      1       l_comment:0.0%          0      98|1   100|0   876b    11k   121  PRI   12:26:47 
    *0      3     *0     *0       0     1|0       0  35.9g   115g  50.5g      0       l_comment:0.0%          0      96|1   100|0   345b     5k   121  PRI   12:26:49 
     1     19      1     *0       4    10|0       0  35.9g   115g  50.5g      2       l_comment:0.0%          0      26|0    44|0     4k    23k   121  PRI   12:26:50 
    *0     *0      1     *0       0     2|0       0  35.9g   115g  50.5g      0       l_comment:0.0%          0       0|0   100|0    62b     3k   121  PRI   12:26:51 
    *0     *0     *0     *0       0     1|0       0  35.9g   115g    48g      0      l_comment:15.8%          0       0|0   100|0    62b     3k   121  PRI   12:26:52 
    *0      2     *0     *0       0     1|0       0  35.9g   115g  48.9g      1       l_comment:0.1%          0       0|0   100|0   267b     4k   121  PRI   12:26:53 
insert  query update delete getmore command flushes mapped  vsize    res faults            locked db idx miss %     qr|qw   ar|aw  netIn netOut  conn repl       time 
    *0     *0     *0     *0       0     1|0       0  35.9g   115g  49.3g      0       l_comment:0.0%          0       0|0   100|0    62b     3k   121  PRI   12:26:54 
    *0     *0     *0     *0       1     1|0       0  35.9g   115g  49.6g      0       l_comment:0.0%          0       0|0   100|0   112b     3k   121  PRI   12:26:55 
    *0      4      3     *0       1     3|0       0  35.9g   115g  49.9g      0       l_comment:0.1%          0       0|0   100|0     1k     4k   121  PRI   12:26:56 
    *0      2     *0     *0       0     1|0       0  35.9g   115g  50.1g      0       l_comment:0.0%          0       0|0   100|0   230b     3k   121  PRI   12:26:57 
    *0      7     *0     *0       0     1|0       0  35.9g   115g  50.2g      0       l_comment:0.0%          0       0|0   100|0   718b     3k   121  PRI   12:26:58 
    *0      4     *0     *0       0     1|0       0  35.9g   115g  50.4g      1       l_comment:0.0%          0       0|0   100|0   472b     3k   121  PRI   12:26:59 
    *0      2     *0     *0       0     1|0       0  35.9g   115g  50.4g      0       l_comment:0.0%          0      95|0   100|0   363b     3k   121  PRI   12:27:00 
    *0      7     *0     *0       1     3|0       0  35.9g   115g  50.4g      0       l_comment:0.0%          0      79|0    92|0     1k     4k   121  PRI   12:27:01 
    *0      4     *0     *0       0     1|0       0  35.9g   115g  50.4g      0       l_comment:0.0%          0      83|0    98|0   509b     4k   121  PRI   12:27:02 
    *0      1     *0     *0       0     1|0       0  35.9g   115g  50.4g      0       l_comment:0.0%          0      96|0   100|0   242b     3k   121  PRI   12:27:03 
insert  query update delete getmore command flushes mapped  vsize    res faults            locked db idx miss %     qr|qw   ar|aw  netIn netOut  conn repl       time 
    *0      4     *0     *0       0     1|0       0  35.9g   115g  50.4g      0       l_comment:0.1%          0      85|0    99|0   429b     4k   121  PRI   12:27:04 
    *0      1      1     *0       1     1|0       0  35.9g   115g  50.5g      0       l_comment:0.0%          0       0|0   100|0   233b     3k   121  PRI   12:27:05 
    *0      5      2     *0       1     3|0       0  35.9g   115g  50.5g      0       l_comment:0.0%          0       0|0   100|0   828b     4k   121  PRI   12:27:06 

below is server status at time 12:25 and 12:26 , I can not find information useful about the problem. limit to body length, I can only paste once of the output.

{ "host" : "mongodb-zhaowei-bj-201-56.chosk.net",
  "version" : "2.6.4",
  "process" : "mongod",
  "pid" : { "$numberLong" : "26229" },
  "uptime" : 13981,
  "uptimeMillis" : { "$numberLong" : "13981035" },
  "uptimeEstimate" : 13871,
  "localTime" : { "$date" : "2014-10-16T12:25:01.407+0800" },
  "asserts" : { "regular" : 0,
    "warning" : 0,
    "msg" : 0,
    "user" : 14025,
    "rollovers" : 0 },
  "backgroundFlushing" : { "flushes" : 233,
    "total_ms" : 381,
    "average_ms" : 1.63519313304721,
    "last_ms" : 2,
    "last_finished" : { "$date" : "2014-10-16T12:25:00.431+0800" } },
  "connections" : { "current" : 102,
    "available" : 2298,
    "totalCreated" : { "$numberLong" : "1037" } },
  "cursors" : { "note" : "deprecated, use server status metrics",
    "clientCursors_size" : 1,
    "totalOpen" : 1,
    "pinned" : 0,
    "totalNoTimeout" : 0,
    "timedOut" : 1 },
  "dur" : { "commits" : 30,
    "journaledMB" : 0.032768,
    "writeToDataFilesMB" : 0.010758,
    "compression" : 2.839268694220605,
    "commitsInWriteLock" : 0,
    "earlyCommits" : 0,
    "timeMs" : { "dt" : 3069,
      "prepLogBuffer" : 0,
      "writeToJournal" : 1,
      "writeToDataFiles" : 0,
      "remapPrivateView" : 0 } },
  "extra_info" : { "note" : "fields vary by platform",
    "heap_usage_bytes" : 68060432,
    "page_faults" : 89955 },
  "globalLock" : { "totalTime" : { "$numberLong" : "13981043000" },
    "lockTime" : { "$numberLong" : "3453729" },
    "currentQueue" : { "total" : 0,
      "readers" : 0,
      "writers" : 0 },
    "activeClients" : { "total" : 0,
      "readers" : 0,
      "writers" : 0 } },
  "indexCounters" : { "accesses" : 772667,
    "hits" : 772666,
    "misses" : 0,
    "resets" : 0,
    "missRatio" : 0 },
  "locks" : { "." : { "timeLockedMicros" : { "R" : { "$numberLong" : "488804" },
        "W" : { "$numberLong" : "3453729" } },
      "timeAcquiringMicros" : { "R" : { "$numberLong" : "691499" },
        "W" : { "$numberLong" : "13208190" } } },
    "admin" : { "timeLockedMicros" : { "r" : { "$numberLong" : "72766" },
        "w" : { "$numberLong" : "0" } },
      "timeAcquiringMicros" : { "r" : { "$numberLong" : "2268880" },
        "w" : { "$numberLong" : "0" } } },
    "local" : { "timeLockedMicros" : { "r" : { "$numberLong" : "714499" },
        "w" : { "$numberLong" : "1422409" } },
      "timeAcquiringMicros" : { "r" : { "$numberLong" : "21035232" },
        "w" : { "$numberLong" : "1946197" } } },
    "l_comment" : { "timeLockedMicros" : { "r" : { "$numberLong" : "259016933" },
        "w" : { "$numberLong" : "8179608" } },
      "timeAcquiringMicros" : { "r" : { "$numberLong" : "284944268" },
        "w" : { "$numberLong" : "8533230" } } },
    "l_recommend" : { "timeLockedMicros" : { "r" : { "$numberLong" : "202152" },
        "w" : { "$numberLong" : "279" } },
      "timeAcquiringMicros" : { "r" : { "$numberLong" : "29442" },
        "w" : { "$numberLong" : "14" } } },
    "lifeix_relation" : { "timeLockedMicros" : { "r" : { "$numberLong" : "81257" },
        "w" : { "$numberLong" : "168" } },
      "timeAcquiringMicros" : { "r" : { "$numberLong" : "30018" },
        "w" : { "$numberLong" : "6" } } } },
  "network" : { "bytesIn" : 64549227,
    "bytesOut" : 525529045,
    "numRequests" : 496002 },
  "opcounters" : { "insert" : 4440,
    "query" : 304833,
    "update" : 18272,
    "delete" : 8,
    "getmore" : 16192,
    "command" : 143477 },
  "opcountersRepl" : { "insert" : 0,
    "query" : 0,
    "update" : 0,
    "delete" : 0,
    "getmore" : 0,
    "command" : 0 },
  "recordStats" : { "accessesNotInMemory" : 8390,
    "pageFaultExceptionsThrown" : 1208,
    "admin" : { "accessesNotInMemory" : 0,
      "pageFaultExceptionsThrown" : 0 },
    "l_comment" : { "accessesNotInMemory" : 8386,
      "pageFaultExceptionsThrown" : 1208 },
    "l_recommend" : { "accessesNotInMemory" : 3,
      "pageFaultExceptionsThrown" : 0 },
    "lifeix_relation" : { "accessesNotInMemory" : 0,
      "pageFaultExceptionsThrown" : 0 },
    "local" : { "accessesNotInMemory" : 1,
      "pageFaultExceptionsThrown" : 0 } },
  "repl" : { "ismaster" : true },
  "writeBacksQueued" : false,
  "mem" : { "bits" : 64,
    "resident" : 619,
    "virtual" : 74035,
    "supported" : true,
    "mapped" : 36785,
    "mappedWithJournal" : 73570 },
  "metrics" : { "cursor" : { "timedOut" : { "$numberLong" : "1" },
      "open" : { "noTimeout" : { "$numberLong" : "0" },
        "pinned" : { "$numberLong" : "0" },
        "total" : { "$numberLong" : "1" } } },
    "document" : { "deleted" : { "$numberLong" : "8" },
      "inserted" : { "$numberLong" : "4440" },
      "returned" : { "$numberLong" : "888831" },
      "updated" : { "$numberLong" : "13597" } },
    "getLastError" : { "wtime" : { "num" : 0,
        "totalMillis" : 0 },
      "wtimeouts" : { "$numberLong" : "0" } },
    "operation" : { "fastmod" : { "$numberLong" : "12462" },
      "idhack" : { "$numberLong" : "0" },
      "scanAndOrder" : { "$numberLong" : "0" } },
    "queryExecutor" : { "scanned" : { "$numberLong" : "1296167" },
      "scannedObjects" : { "$numberLong" : "623453" } },
    "record" : { "moves" : { "$numberLong" : "0" } },
    "repl" : { "apply" : { "batches" : { "num" : 0,
          "totalMillis" : 0 },
        "ops" : { "$numberLong" : "0" } },
      "buffer" : { "count" : { "$numberLong" : "0" },
        "maxSizeBytes" : 268435456,
        "sizeBytes" : { "$numberLong" : "0" } },
      "network" : { "bytes" : { "$numberLong" : "0" },
        "getmores" : { "num" : 0,
          "totalMillis" : 0 },
        "ops" : { "$numberLong" : "0" },
        "readersCreated" : { "$numberLong" : "0" } },
      "preload" : { "docs" : { "num" : 0,
          "totalMillis" : 0 },
        "indexes" : { "num" : 0,
          "totalMillis" : 0 } } },
    "storage" : { "freelist" : { "search" : { "bucketExhausted" : { "$numberLong" : "0" },
          "requests" : { "$numberLong" : "10477" },
          "scanned" : { "$numberLong" : "16271" } } } },
    "ttl" : { "deletedDocuments" : { "$numberLong" : "0" },
      "passes" : { "$numberLong" : "233" } } },
  "ok" : 1 }
  • Funny question to ask with that name! I'll answer below! – Markus W Mahlberg Oct 17 '14 at 6:45
  • It's memory (not memery) – marc_s Oct 17 '14 at 9:30
  • 1
    Can you post db.serverStatus() output for this mongod instance? – James Wahlin Oct 17 '14 at 12:48
  • 2
    Your resident is exceeding your mapped, and that is very odd. What OS is this, and what version of MongoDB? I'd usually point you at mongomem (eng.wish.com/mongomem-memory-usage-by-collection-in-mongodb) but that's more useful when things are behaving more normally. You have resident memory going up without any hard faults, and more data in res mem than exists on disk. Still, it might be interesting to see the output – Adam C Oct 17 '14 at 17:39
  • Thanks for reply , I had append one of the serverStatus() output . – Malloc Oct 18 '14 at 8:20
0

MongoDB makes good use of your RAM. As long there is free RAM, it will take it. It will first try to get the indices into RAM. You can see their size with

use <yourDB>
db.status()

from the mongo shell.

Next it will try to keep as much as possible of your working set in RAM (which is a bit simplified, but sufficient for this explanation).

MongoDB does all this to improve performance, which is a good thing. It does not make much sense to have free RAM I'd,Ingrid around because it might be possible that some other process might need it in the future. It is much better to utilize the RAM available and free it on demand. Which is a bit of the discussion between Windows and UNIX memory usage patterns. Judging from that, I'd assume you run MongoDB on a Windows machine ;).

The spikes you are experiencing usually happen under more or less high load. As long as there is no impact on other processes, there is nothing to worry about. If other processes are impacted, I'd strongly suggest migrating them or MongoDB to a dedicated machine/instance/VM. Please read the MongoDB Production Notes for details on how to set up optimal machines for MongoDB.

  • I know some about the unix memory usage patterns, but this is not normal memory used. The mongodb worked well when the number of column "res" keep about 1.6g, but it raised up to 64g in about 2 minutes.But my working set is much smaller than 60g. When use of "res" raised, there is a lot of connection timeout from client application log file, so it's not normal. And I can not find useful thing about the cause. The machine is just for mongodb, there is no other process need much resource. – Malloc Oct 17 '14 at 7:25
  • You are prone to a fallcy here: post hoc, ergo propter hoc. Memory usage may well rise because of a lot of connections are opened, costly aggregations and/or MRs are done, etc. So those new connections themselves could be the problem. I'd rather investigate this end. The fact that it is a spike clearly shows that we don't have a memory leak here. So I'd investigate what usage is brought to the server during that time. – Markus W Mahlberg Oct 17 '14 at 7:35
  • There is no new connections, just old connections timeout. mongostat shows that querys/s is decreasing during the 2 minutes, because most of the connections is timeouted. The server can not work until be restarted. – Malloc Oct 17 '14 at 9:46

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.