Copy Specific Columns from DB 1 To DB 2 on MongoDB

Question

I have Two Instance of Node JS server which is handled By NGINX. First server is connect with MongoDB First Server i.e

mongoose.connect('mongodb://127.0.0.1/db_primary', { useMongoClient: true });

While Second Node Js Server is Connect with MongoDB Second Serer i.e

mongoose.connect('mongodb://127.0.0.1/db_secondary', { useMongoClient: true });

Actually, I have to Do some Heavy Calculations on DB and that the main reason I will divide my DB into Two Pieces.

Initially When I have One Server then if the backend is busy with calculations (Millions of Entries) at that time Server is completely freeze. So by the Separation There is now no freezing point of server.

My Question is How I can Copy the selected Columns on The entire Table form DB 1 And paste it to DB 2 with only Selected Columns.

Example:Database 1 


{
    _id: ObjectID(54815dfsfsd1f564168ad51a5s),
    a: 123,
    b: 768,
    c: 89.67,
    d: 8976,
    e: 45
  }..........upto 10 million entries

Now I want to Copy Only Column c & e from Database 1 to Database 2.

Expected Result: 
{
        _id: ObjectID(54815dfsfsd1f564168ad51a5s),
        c: 89.67,
        e: 45
}..........upto 10 million entries

And all these things are done by CronJob of Nodejs. I mean Not a manual task.

I am Newbie on MongoDB. Any help is appreciated.

Answer 1

You can do projection with query. So, db.collection.find({},{c:1,e:1}) and then insert that to second DB.

Inside the one DB you can do it with use of aggregation with project and out stages.

Have you consider to use replica set and secondary read?

Answer 2

Now I want to Copy Only Column c & e from Database 1 to Database 2.

It's Only possible through mongo shell method like cursor.forEach() & db.getSiblingDB(). you can copy the one database collection some selected documents to the another database collection.

My Question is How I can Copy the selected Columns on The entire Table form DB1 And paste it to DB2 with only Selected Columns.

Let's start from beginning , suppose that you have two database db1 & db2.

use db1
switched to db db1

I am going to create collection in db1 having name orders

> db.createCollection("order")
{ "ok" : 1 }

Again i am going to insert some documents in orders collection in db1.

> db.orders.insertMany([
...  {       "_id" : ObjectId("5a44c1479adf6e5fc5cea525"),
...         "cust_id" : "A123",
...         "amount" : 500,
...         "status" : "A"
... },
... {
...         "_id" : ObjectId("5a44c1479adf6e5fc5cea526"),
...         "cust_id" : "A123",
...         "amount" : 250,
...         "status" : "A"
... },
... {
...         "_id" : ObjectId("5a44c1479adf6e5fc5cea527"),
...         "cust_id" : "B212",
...         "amount" : 200,
...         "status" : "A"
... },
... {
...         "_id" : ObjectId("5a44c1479adf6e5fc5cea528"),
...         "cust_id" : "A123",
...         "amount" : 300,
...         "status" : "D"
... }])
{
        "acknowledged" : true,
        "insertedIds" : [
                ObjectId("5a44c1479adf6e5fc5cea525"),
                ObjectId("5a44c1479adf6e5fc5cea526"),
                ObjectId("5a44c1479adf6e5fc5cea527"),
                ObjectId("5a44c1479adf6e5fc5cea528")
        ]
}

In the above collection orders there is 4 documents , But i only want to copy some field records ( Suppose that here i want to copy "cust_id" : "B212" respective records to db2).

Again i am going to create second database db2.

> use db1
switched to db db2

Again i am going to create the collection orders1 in db2.

> db.createCollection("order1")
{ "ok" : 1 }

Note : Here for running the below mention query you have to switch to db1. because we are going to copy the collection records from db1 to db2.

> use db1
switched to db db1

Then execute the below mention query

db.orders.find({"cust_id": "B212"}).forEach(function(d){db.getSiblingDB('db2')['orders1'].insert(d);});

For confirmation of copy of that records field has successfully done or not . You have to switch to db2. And find out the collection order1 records in db2.

> use db2
switched to db db2

> show collections
orders1

> db.orders1.find()
{ "_id" : ObjectId("5a44c1479adf6e5fc5cea527"), "cust_id" : "B212", "amount" : 200, "status" : "A" }
>

Or through below command to find the records in proper format

> db.orders1.find().pretty()
{
        "_id" : ObjectId("5a44c1479adf6e5fc5cea527"),
        "cust_id" : "B212",
        "amount" : 200,
        "status" : "A"
}
>

As above i have already said that only i want to copy the "cust_id" : "B212" records from db1->orders to db2->orders1.

Note: Here the two databases would need to share the same mongod for this to work.

For your further ref Here and Here