Questions tagged [csv]

Questions on how to import into, extract data from, or exchange data between databases in CSV format (plain text using commas to separate values).

Filter by
Sorted by
Tagged with
2
votes
2answers
872 views

Importing from CSV file, is crashing mysql server

I am facing a problem with importing large data into single mysql table. Basically I assume it is happening because limited machine resources but, I am having troubles to proof it. this is my CSV ...
0
votes
1answer
2k views

Single value from a list of comma separated values when used in inner join in SQL Server

Can anyone please help me with a query to return a single value from a list of comma separated values when used in inner join. I can paste the example query here. select * from Test.dbo.Test1 ...
0
votes
0answers
767 views

Export Stored Procedure Results To CSV

I have seen many options for doing such, however most of them involve using bcp or using exec master..xp_cmdshell which I am unable to use on my server (security restrictions). I was looking at ...
3
votes
1answer
671 views

File size when extracting from SQL Server to CSV

I am attempting to extract a table to CSV using BCP from SQL Server 2014 and the size of the file once the data is extracted is actually larger than the size of the database itself. One thing to note ...
10
votes
2answers
12k views

How can I dump all tables to CSV for a PostgreSQL schema?

I have a database with a lot of schemas in it and I want to dump the each of the table contents to CSV. I'm aware of the COPY command but I'm not sure how to script something that will read all of ...
1
vote
1answer
4k views

Can I load a CSV into a stored procedure that inserts into multiple tables?

OK, an example is often easier to follow, so here is my procedure: delimiter // CREATE PROCEDURE mysql.insertTables (_first_name varchar(30), _last_name varchar(45), _create_time timestamp, ...
3
votes
1answer
2k views

Syncing a SQL Server table with a CSV file

I have recently started a project with sensors that add data to a single CSV file as data the data. I would like to have this data stored in a SQL Server database in a relatively real-time manner. I ...
4
votes
2answers
7k views

Does the PostgreSQL COPY command have the option of choosing with fields to map the CSV columns to?

Does the PostgreSQL COPY command have the option of choosing with fields to map the CSV columns to? The PostgreSQL COPY command appears to expect that the target table matches its columns exactly. Am ...
5
votes
1answer
11k views

Mysql LOAD DATA INFILE 100,000 csv records perfect BUT 300,000 records - 0 rows inserted

The following query imports 100,000 perfectly, but larger files nothing happens. No errors, no rows loaded. Any clues you can offer are appreciated. I spent hours looking into max file and memory ...
4
votes
2answers
2k views

Directly import a csv gzip'ed file into SQLite 3

I'd like to import a 15GB file comma delimited gzip compressed file into Sqlite 3 without having to use temporary files. I'd like to run a command like: zcat input/surgical_code.csv.gz | tail -n +...
2
votes
1answer
1k views

Postgres \copy from CSV: how to format range variables?

Consider a Postgres table containing a range variable: CREATE TABLE my_table (t_range tsrange, field1 text); If I want to manually insert a row via psql, I can run: INSERT INTO my_table (t_range, ...
1
vote
1answer
45 views

Check value when I have a list of elements separated by commas in a column?

I am looking to return a value in a database. Table named TIPOLOGY is structured like this: Name Owner ID Office Type LIST_OF_VALUES Jones Lasi 1 12345 TKT AUD,PAX,CCF ...
0
votes
1answer
1k views

Migrating data from Firebird to CSV on OSX

I have a .gdb Firebird database file that I would like to partially transform to .csv. From the CLI tool, I'm able to do the following: CONNECT "/Users/jpmonette/Desktop/file.gdb"; OUTPUT /Users/...
4
votes
1answer
4k views

MySql Workbench Data Import .csv

Hi I am trying to use to MySql Workbench Data Import facility to import a .csv file (thousands of records). However it stops importing the first 5 records. After studying the .csv file I could see ...
0
votes
1answer
51 views

question regarding csv format?

CSV format cannot be used for importing text files to a database unless the table for the data is already defined in the database. is this true or false?
3
votes
1answer
294 views

Update foreign key from csv file

I am trying to set up a simple database for the first time using MySQL Workbench. I'd like to update the foreign keys of the child table based on the values of parent_child.csv. Here are the current ...
1
vote
2answers
7k views

Load csv file to mysql using load data local infile

I need a simple way to load a csv file from my computer to the database (which is on a remote server). I'm trying to load the file using a php page that executes this code: $file = $_FILES['csv']['...
0
votes
1answer
3k views

Line-breaks in Text column of CSV file

I have come across an awkward situation where I have got CSV files with | as the column delimiter. In these files, I don't have Text column qualifier. (i.e. "). My problem is when I am importing these ...
0
votes
1answer
305 views

Import quoted CSV

I have a set of quoted CSV files, which I need to import into SQL Server. Files look like this: "ColA","ColB","ColC" "12345","John Smith","-75.33" "12356","Jane Doe","43.12" BCP and BULK INSERT ...
0
votes
1answer
356 views

Fill a GTT with external data each session

I have to import an external csv file into a table in database each day. The file is encoded in 'UCS2-LE BOM'. I have found a solution using external tables and bulk insertion, but the problem is to ...
1
vote
1answer
617 views

leading 0 missing in column when exporting to csv

I have a query I export which is then used to import products into our website. One of the ways we help people find products more easily is by using a long and short item code. It's basically the ...
0
votes
2answers
3k views

problems when importing csv files into SQL Server using SSIS when the columns are not consistent from quarter to quarter

I am using SSIS to upload a bunch of csv tables into SQL Server Express. The csv files are quarterly data files. They are largely consistent but every couple of years a column may be added or an ...
0
votes
1answer
295 views

SSIS - Update Specific Cell in CSV

I have a pre-defined csv template. The header row begins on row 5. Prior to this I have a date value in cell B2. How do I update this cell value in SSIS?
2
votes
3answers
1k views

Best DB structure for large CSVs?

I have previously posted on stackoverflow, however, I have been directed here. I'm getting to the stage where I need to analyse lots of flat CSVs: Read-in around 500GB of CSVs (e.g. daily data split ...
0
votes
1answer
552 views

Load data - fields terminated by character appearing within a field statement

I have an issue with loading data from a csv file. The question is, without preprocessing, is it possible to have an exception in the following code load data local infile 'Program.csv' into table ...
0
votes
0answers
861 views

Restore .dmp file into csv

We have asked our clients for certain data in a csv file format and we just got a .dmp file. I have been trying to open it using MySQL as if it were .dump but it doesn't work. I have seen other ...
3
votes
2answers
11k views

How to create and populate a PostgreSQL table from CSV automatically?

I am new to database management and we are using PostgreSQL. All I need to do is to migrate CSV files (corresponding to around 200 tables) to our database. Manually creating tables for every CSVfile ...
2
votes
2answers
15k views

MySQL LOAD DATA INFILE Error Code 1366. Incorrect integer value: ' '

I've been trying to upload a CSV table into my remote server database but I find an error. I use MySQL in a RedHat Linux distrib. My mycsv.csv file looks like this: aa,ProductDescription,country,...
3
votes
1answer
1k views

SSIS Repeated Header In CSV File

I am writing to a csv file in a Foreach loop. For each loop the code is writing the header so it appears repeatedly throughout the file. How do I ensure the header only appears once? I have tried ...
1
vote
1answer
3k views

Import CSV to MySQL with long text fields containing line breaks

I'm trying to import a CSV file (';' separated) to a MySQL table. Normaly, I use a SQL like: LOAD DATA LOCAL INFILE 'local-path/file.csv' INTO TABLE tmp_table CHARACTER SET latin1 FIELDS TERMINATED ...
1
vote
2answers
6k views

HeidiSQL: Import CSV

A few lines of my CSV: "id","date","date_","pow","ene","reac" "ES0134000563024539AM0F","2013-05-02","2013-06-05","62","43","43" "ES0134000573024539AM0F","2013-05-02","2013-06-05","62","43","43" "...
2
votes
0answers
441 views

pgAdmin export line overflow on large field

Background: I have a database of significant geographical locations downloaded from GeoNames open database. The create script for the table is as follows CREATE TABLE populargeonameswiki ( geonameid ...
1
vote
3answers
8k views

SSIS: How to check if a record DOESN'T exist in flat file but exists on the database

I am working on preparing an SSIS job where I am importing a .CVSV file to OLE DB destination (sql database). We are going to get these files on daily basis. The .CSV file contains records of doctors. ...
2
votes
2answers
3k views

Importing CSV's into MySQL(5.6) tables leaves date fields as 0000-00-00

During the import from a CSV file created from an MS Access table into a MySQL table it converts all date fields in the csv into 0000-00-00. I use the following Load Data command for the import. ...
3
votes
1answer
16k views

import CSV into mysql table via command prompt (MySQL 5.6.25)

I'm trying to load CSV's into a mysql database with this command: LOAD DATA LOCAL INFILE '\Users\userName\Downloads\tableName.csv' INTO TABLE tableName FIELDS TERMINATED BY ',' LINES TERMINATED BY ...
4
votes
2answers
14k views

Copying CSV file to temp table with dynamic number of columns?

I am wondering if there is a way to copy a csv file into a temp table where the number of columns is unknown in the csv file. The DB software I'm using is PgAdmin III. I figured out that if I do know ...
4
votes
5answers
9k views

Bulk Insert - (0 row(s) affected)

I am trying to import a .CSV file into a mssql table using bulk insert (Although I am open to other methods). I have a bulk insert sql statement but it says (0 row(s) affected) when I execute the the ...
1
vote
0answers
1k views

Why is there a difference in CSV Export Results of Oracle SQL Developer and SQL Plus Spool

When I export data from SQL Developer GUI, setting format: csv, left enclosure: ", right enclosure: ", and Encoding: UTF8, I get data exactly the way that I need it for processing in Python. However,...
2
votes
1answer
2k views

Compact Spool file from Sqlplus

I have scripts that I run in Oracle SQL Developer and then export the result to my pc as a csv. I now started using sqlplus with a .bat to run my scripts and spool the result to my pc. But I noticed ...
1
vote
1answer
8k views

Correct 'lines terminated by' for query

I am trying to query two tables of data and send them to a csv file for upload to a website. However i am having trouble getting the formatting to play ball. I want headings to separate into columns ...
6
votes
3answers
6k views

SQLite CSV import won't create a table automatically

According to https://www.sqlite.org/cli.html#csv, sqlite> .mode csv sqlite> .import C:/work/somedata.csv tab1 There are two cases to consider: (1) Table "tab1" does not previously exist ...
0
votes
1answer
490 views

How to get line that causes error when loading from csv to MySQL

I'm trying to load data from CSV file, but occurs error Error Code: 1366. Incorrect integer value: 'abcdxxx' for column 'col' at row 123545. I Have file with multiple breaklines CRLF's in content, ...
0
votes
1answer
219 views

merge multiple fields of csv file while running mysqlimport command

I have a mysql table like this +----------------+--------------+------+-----+---------+----------------+ | Field | Type | Null | Key | Default | Extra | +----------------+---...
3
votes
2answers
18k views

MySQL Error when importing CSV with empty fields

I have CSV file with exported table, that has rows with fields that do not contain values. Columns are separated using "," and if fields have no values they look like that: ,,,,,, I tried to load data ...
1
vote
1answer
3k views

Storing multiple IDs in column or in new table

I'm fairly new to the databases and I'm now creating little project where I have a database and I want to store deck of cards into the table where each card has its own ID (for example 52 cards in a ...
1
vote
1answer
1k views

Reverse auto_increment id existing column in phpMyAdmin

I have imported csv file through phpMyAdmin into one table. I have set auto-increment field, as shown below, but of course it ordered it ascending: +----+------------+------+ | id | date | name ...
1
vote
1answer
298 views

Uber-wide tables: Time to switch to Mongo?

As a web developer, I've typically only worked with nice, normalized tables in SQL-based RDBMS, but I'm doing some data consolidation for a researcher who will be analyzing correlations between all ...
7
votes
4answers
25k views

Import CSV or JSON file into DynamoDB

I have 1000 CSV files. Each CSV file is between 1 and 500 MB and is formatted the same way (i.e. same column order). I have a header file for column headers, which match my DynamoDB table's column ...
9
votes
3answers
20k views

Create a table from a CSV file with headers

I'm seeking to find a way to generate a new MySQL table solely based on the contents of a specified CSV. The CSV files I'll be using have the following properties; "|" delimited. First row specifies ...
2
votes
1answer
7k views

Error message when importing data into SQL (Import & Export Data Wizard)

I am trying to use the Import & Export Data Wizard to import the contents of a CSV file into a table in MSSQL. I have no issues if I import all columns as strings however, I need one column to be ...