0

I'm trying to dump a MongoDB collection from a remote server to a local JSON file.

First, I do some ssh port forwarding to begin with:

ssh -NL 12345:localhost:27017 $REMOTE_USERNAME@$REMOTE_HOST &

Then, the export itself:

mongoexport -h $MONGODB_URL -d $MONGODB_DBNAME -c $MONGODB_COLLECTION -o $BACKUP_FILE --pretty

What I would normally get is something like this:

2019-02-07T14:58:15.322-0400    writing bubeIO.requests to
2019-02-07T14:58:18.754-0400    [........................]  bubeIO.requests  0/2773  (0.0%)
2019-02-07T14:58:21.754-0400    [........................]  bubeIO.requests  10/2773  (0.3%)
2019-02-07T14:58:24.754-0400    [........................]  bubeIO.requests  20/2773  (0.6%)
...
...
2019-02-07T14:59:00.754-0400    [........................]  bubeIO.requests  2773/2773  (100.0%)

But, what I'm getting is this:

2019-02-07T14:58:15.322-0400    writing bubeIO.requests to
2019-02-07T14:58:18.754-0400    [........................]  bubeIO.requests  0/2773  (0.0%)
2019-02-07T14:58:21.754-0400    [........................]  bubeIO.requests  0/2773  (0.0%)
2019-02-07T14:58:24.754-0400    [........................]  bubeIO.requests  0/2773  (0.0%)
2019-02-07T14:58:27.753-0400    [........................]  bubeIO.requests  101/2773  (3.6%)
2019-02-07T14:58:30.753-0400    [........................]  bubeIO.requests  101/2773  (3.6%)
2019-02-07T14:58:33.754-0400    [........................]  bubeIO.requests  101/2773  (3.6%)
2019-02-07T14:58:36.753-0400    [........................]  bubeIO.requests  101/2773  (3.6%)
2019-02-07T14:58:39.753-0400    [........................]  bubeIO.requests  101/2773  (3.6%)
2019-02-07T14:58:42.753-0400    [........................]  bubeIO.requests  101/2773  (3.6%)
2019-02-07T14:58:45.754-0400    [........................]  bubeIO.requests  101/2773  (3.6%)
2019-02-07T14:58:48.753-0400    [........................]  bubeIO.requests  101/2773  (3.6%)
2019-02-07T14:58:51.753-0400    [........................]  bubeIO.requests  101/2773  (3.6%)
2019-02-07T14:58:54.754-0400    [........................]  bubeIO.requests  101/2773  (3.6%)
2019-02-07T14:58:57.753-0400    [........................]  bubeIO.requests  101/2773  (3.6%)
2019-02-07T14:59:00.594-0400    [........................]  bubeIO.requests  101/2773  (3.6%)
2019-02-07T14:59:00.594-0400    Failed: error writing data for collection `bubeIO.requests` to disk: error reading collection: read tcp [::1]:61276->[::1]:12345: use of closed network connection

As you can see, it goes from 0 to 101 real quick and then it doesn't go anywhere from there until timeout.

It worked nicely before for remote dumps and suddenly this starts to happen. Local dumps work just fine, so this could be a networking issue?

Can anyone give me a direction on how to solve this?

PS: mongod and mongoexport are both version 4.0.2

EDIT

It started working again after I freed a total of 13 GB hd space. Maybe the disk could not hold the whole uncompressed collection, so I'm getting deeper in mongoexport's inner workings and disk usage. Thanks to @priyanka-kariya and @Stennie.

Improve this question
3
  • Hi @PriyankaKariya, I did it and the disk had only 3 GB of free space. I cleaned it a little and now the dump is working ok, so thank you a lot! Interesting thing is the collection that I'm dumping is only 61 MB.
    – Victor Valente
    Feb 8, 2019 at 13:17
  • @VictorValente, Did you try to give the some "records" limits of collection during the mongoexport time. for example ( mongoexport --host localhost --db test --collection foo | head -100 > a.json)
    – Md Haidar Ali Khan
    Feb 8, 2019 at 15:59
  • 101 documents (or up to 1MB of results) is the default size for the first batch of results in a query. Subsequent operations use larger batches (up to the 16MB message size limit). This looks like a networking error or timeout with your port forwarding, but there isn't enough information to be sure. The error happens after about 45 seconds, which seems very short for a timeout. What type of remote deployment are you connecting to (standalone, replica set, or sharded cluster)? If you are using a replica set, are you including replicaSet in the connection string?
    – Stennie
    Feb 10, 2019 at 20:46

1 Answer 1

Reset to default
0

Please check the disk space on target machine

Improve this answer
0

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.