0

I am attempting to load S3 files to an Aurora MySQL DB Cluster using the below AWS documentation:

https://docs.aws.amazon.com/AmazonRDS/latest/AuroraUserGuide/AuroraMySQL.Integrating.LoadFromS3.html

LOAD DATA FROM S3 PREFIX '://' INTO TABLE my_table FIELDS TERMINATED BY ',' LINES TERMINATED BY '\n';

Everything works fine and looks good in MySQL when I run a test on 1-2 csv files in my S3 bucket, but when I try loading 2,000+ csv files from my S3 bucket, after exactly 7,200 seconds the query fails and I get the error "MySQL Error 2013 Lost connection to MySQL server during query" (this has happened multiple times now).

MySQL settings are set as such:

MySQL_Settings

I am using MySQL Workbench, MySQL Version 8.0.15, and engine InnoDB.

Any ideas why this keeps happening? Any guidance or help would be extremely appreciated!!

  • Just an update, looks like it is always uploading 98% of all the rows from the S3 bucket into the DB Cluster, but always fails at 7,200 seconds. One upload was 40 million rows and another was 73 million but it always took 7,200 seconds regardless of the amount of rows it was uploading. – Samer Mar 26 at 15:57
  • Look at the instance level values in the db parameter group to see if there is a value of ~7200 for connection_timeout, lock_wait_timeout,long_query_time or wait_timeout. docs.aws.amazon.com/AmazonRDS/latest/AuroraUserGuide/… – Aaron Mar 26 at 16:52
  • @Aaron Thanks, I checked all those parameters and did not find anything with a value of ~7200. Also, none of those parameter groups are selected as part of my parameters :( – Samer Mar 26 at 19:37
  • SHOW VARIABLES WHERE value BETWEEN 6666 and 9999; – Rick James Apr 18 at 6:15

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Browse other questions tagged or ask your own question.