0

I am working on a requirement to transfer almost 1 TB of data from a CSV file to SQL server. Which is the fastest way to do this? I am considering bulk copy and came across this link which actually has a power shell script for the same

https://gallery.technet.microsoft.com/scriptcenter/Import-Large-CSVs-into-SQL-216223d9

Are there any other considerations which I am missing? The source DB is a PeopleSoft Oracle database.

Another option I considered was using the SSMA tool for Oracle, copy the entire schema to SQL Server and then query the required data from the database. Is this a better option? Please suggest

Thanks!

  • I always like community scripts as it has many eyes. You'll have to test this one. My opinion is I would try to do it from a good loop and track what rows got imported in big chunks from within sql server directly and make sure I manage memory correctly. Note the script has a 'turbo' mode which locks the table, you could utilize the same actions. This script does it through PS which is fine as long as it doesn't run out of memory which it won't if it is written to loop through correctly. It has lots of eyes on it which is good too. Can you try to import some rows and see the impact? – Ali Razeghi Dec 6 '16 at 0:40
  • I tried extracting some of the rows and saved the csv as a comma delimiter and it failed. What is the best way to extract it in CSV? Should I try with Tab delimiter? Please advice – user2923332 Dec 9 '16 at 18:19

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Browse other questions tagged or ask your own question.