Let's say I have multiple columns on sql server that I want to join and save the resultset into a single mongodb collection for faster lookup and calculations ( I'm talking about over 15 tables and more than a million resultset).

Is there any faster way to perform other than using a programming language to query the SQLserver db and then upsert into mongo?

  • Just a heads up, MongoDB isn't inherently any faster than SQL Server at lookups or calculations, it just depends how you structure things in either system. As far as migrating the data into MongoDB, I'm not sure of a specific tool, but I know external systems (such as AWS) generally leverage SQL Server's Change Data Capture (CDC) feature. – J.D. Mar 27 at 22:53
  • Thanks for you reply. I know that mongo isnt inherently any faster than SQL Server, its just that some parts of our application consumes queries that cost way to much CPU, so we want to save the resultset of this query in specific in mongo so the app can consume data from mongo instead of sqlserver ( i could have saved this resultset in sql server aswell but i wanted to free CPU usage of our sql server machine). I already perform this ETL by consuming the query and sending it to mongo via nodeJS, but i want to know if theres any faster way to do this – beto Mar 27 at 23:06
  • NP! It just depends on how you need to structure the data, but it sounds like currently it's not structured how you need. So I think no matter what you're unfortunately going to have to use some CPU to structure it as needed, in which case if you're already performing the ETL on it to structure it how you want to consume it in your application, then you can just as well store it in another table in SQL Server, as a simple lookup on a table uses no measurable CPU. The only other idea I have is to export the data as it is and then structure it in the external system, essentially ELT. – J.D. Mar 27 at 23:14

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Browse other questions tagged or ask your own question.