I have been trying to import CSV data about 60 MB; it contains about 400,000 rows, which should not be too much. I tried these ways to import the data:
- MySQL Workbench - it took about two hours!!!
- Using MySQL
LOAD INFILEcommand - it took about 15 seconds!
- Using Talend Open Studio - almost as long as Workbench.
Why are the GUI tools working so badly? I want to use GUI-based tools, instead of writing scripts.
With Talend Open Studio, is there a way to speed things up? I heard that there is possibly another Talend product that deals with Big Data, will this do it faster than Talend Open Studio?
Hope someone out there knows!
I know that in Hadoop using Pig, I can get the the CSV from HDFS into Pig memory very quickly.