I'm trying to import a large file, around 2GB, into MariaDB Columnstore. Other files that are smaller worked fine, so I suspect it's a size issue. I create the table with engine Columnstore with no problem. Then I run the command:

sudo cpimport -s '|' -E '"' industry  fdic_callr_rcr_regulat_capital fdic_callr_rcr_regulat_capital.csv

And I get:

Using table OID 20208 as the default JOB ID
Input file(s) will be read from : /home/sofuser/data/callr
Job description file : /var/lib/columnstore/data/bulk/tmpjob/20208_D20210617_T180941_S792633_Job_20208.xml
Log file for this job: /var/lib/columnstore/data/bulk/log/Job_20208.log
2021-06-17 18:09:42 (2870) INFO : successfully loaded job file /var/lib/columnstore/data/bulk/tmpjob/20208_D20210617_T180941_S792633_Job_20208.xml
2021-06-17 18:09:42 (2870) INFO : Job file loaded, run time for this step : 0.40599 seconds
2021-06-17 18:09:42 (2870) INFO : PreProcessing check starts
2021-06-17 18:09:42 (2870) INFO : input data file /home/sofuser/data/callr/fdic_callr_rcr_regulat_capital.csv
2021-06-17 18:09:50 (2870) ERR  : Error opening column file for OID-21217; DBRoot-1; partition-0; segment-0; filename-/var/lib/columnstore/data1/000.dir/000.dir/082.dir/225.dir/000.dir/FILE000.cdf;  opening a column file. The file was not found or was inaccessible. [1052]
2021-06-17 18:09:50 (2870) CRIT : Error in pre-processing the job file for table industry.fdic_callr_rcr_regulat_capital [1052]
2021-06-17 18:09:50 (2870) ERR  : Error deleting meta file; Error deleting bulk rollback data subdirectory /var/lib/columnstore/data1/bulkRollback/20208_data; [1053]

NOTE: the file that's not found is not the input file, it's a file that MariaDB has in its database structure.

This is the log

  • A question for you: how much storage space remains on the device where /var/lib/columnstore resides?
    – matigo
    Jun 21 '21 at 8:11
  • There's only one partition that's 128GB used 9%. The CSV file size is 2GB.
    – ps0604
    Jun 21 '21 at 14:16
  • seems like file is corrupted & can share the server configuration ? Jun 25 '21 at 2:31
  • That's something I haven't tried, to recreate the file
    – ps0604
    Jun 25 '21 at 2:49

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Browse other questions tagged or ask your own question.