2

Trying to insert several JSON files to MongoDB collections using shell script as following,

#!/bin/bash

NUM=50000
for ((i=o;i<NUM;i++))
do
    mongoimport --host localhost --port 27018 -u 'admin' -p 'password' --authenticationDatabase 'admin' -d random_test -c tri_${i} /home/test/json_files/json_${i}.csv --jsonArray
done  

after several successful adding, these errors were shown on terminal

Failed: connection(localhost:27017[-3]), incomplete read of message header: EOF
error connecting to host: could not connect to server: 
server selection error: server selection timeout, 
current topology: { Type: Single, Servers: 
[{ Addr: localhost:27017, Type: Unknown, 
State: Connected, Average RTT: 0, Last error: connection() : 
dial tcp [::1]:27017: connect: connection refused }, ] }

And below the eoor messages from mongo.log, that said too many open files, can I somehow limit the thread number? or what should I do to fix it?? Thanks a lot!

2020-07-21T11:13:33.613+0200 E  STORAGE  [conn971] WiredTiger error (24) [1595322813:613873][53971:0x7f7c8d228700], WT_SESSION.create: __posix_directory_sync, 151: /home/mongodb/bin/data/db/index-969--7295385362343345274.wt: directory-sync: Too many open files Raw: [1595322813:613873][53971:0x7f7c8d228700], WT_SESSION.create: __posix_directory_sync, 151: /home/mongodb/bin/data/db/index-969--7295385362343345274.wt: directory-sync: Too many open files
2020-07-21T11:13:33.613+0200 E  STORAGE  [conn971] WiredTiger error (-31804) [1595322813:613892][53971:0x7f7c8d228700], WT_SESSION.create: __wt_panic, 490: the process must exit and restart: WT_PANIC: WiredTiger library panic Raw: [1595322813:613892][53971:0x7f7c8d228700], WT_SESSION.create: __wt_panic, 490: the process must exit and restart: WT_PANIC: WiredTiger library panic
2020-07-21T11:13:33.613+0200 F  -        [conn971] Fatal Assertion 50853 at src/mongo/db/storage/wiredtiger/wiredtiger_util.cpp 414
2020-07-21T11:13:33.613+0200 F  -        [conn971]

***aborting after fassert() failure
0

Depending on your OS, there's multiple ways to modify the maxopenfile of MongoDB.
On Debian and Ubuntu, in /lib/systemd/system/mongod.service there's the LimitNOFILE argument which can be increased if needed.

You have more infos on other OS on https://docs.mongodb.com/manual/reference/ulimit/

2
  • I am using redhat, but I don't have the root permission. Can I some how shut down a file once its importing is done? – Erwin Zangwill Jul 31 '20 at 6:05
  • Without the root or sudo permission I don't think you'll be able to change the ulimit either on the OS or the daemon's side unfortunately. All the open files are managed by the mongod process, it's likely it's opening creating files while importinng your data. Have you tried limiting your NUM variable to a much lower value? – bebu Aug 3 '20 at 9:15

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.