A message from our CEO about the future of Stack Overflow and Stack Exchange. Read now.

Questions tagged [csv]

Questions on how to import into, extract data from, or exchange data between databases in CSV format (plain text using commas to separate values).

Filter by
Sorted by
Tagged with
3
votes
1answer
708 views

File size when extracting from SQL Server to CSV

I am attempting to extract a table to CSV using BCP from SQL Server 2014 and the size of the file once the data is extracted is actually larger than the size of the database itself. One thing to note ...
3
votes
1answer
2k views

Syncing a SQL Server table with a CSV file

I have recently started a project with sensors that add data to a single CSV file as data the data. I would like to have this data stored in a SQL Server database in a relatively real-time manner. I ...
3
votes
1answer
316 views

Update foreign key from csv file

I am trying to set up a simple database for the first time using MySQL Workbench. I'd like to update the foreign keys of the child table based on the values of parent_child.csv. Here are the current ...
2
votes
6answers
3k views

Export data from a CCURE 800/8000 System using a pair of ODBC data sources?

I'm working on a security system called CCURE 800/8000 and trying to export some entrance / exit data from the Activity Monitor that logs when people use their badges to enter and exit the building. ...
2
votes
1answer
16k views

Import Data from 48 GB csv File to SQL Server

I am importing huge data file of size around 48 gb using SQL Server default Import Tool. It keeps on executing fine for app. 13000000 Row insertions but after it the Task Fails with following error. ...
2
votes
1answer
3k views

Oracle SQL Developer: Set the <null> value when exporting to CSV

When exporting to CSV in Oracle SQL Developer, null values are replaced with "". I want them to be exported as the string null. Is this configurable anywhere? Example of what is wished: "STARTDATE"...
2
votes
1answer
9k views

Correct 'lines terminated by' for query

I am trying to query two tables of data and send them to a csv file for upload to a website. However i am having trouble getting the formatting to play ball. I want headings to separate into columns ...
2
votes
2answers
887 views

How can I get PostgreSQL to not advance sequences when COPY fails?

I'm importing CSV files into a relatively simple PostgreSQL database. (The CSVs are sometimes created manually from information in a book, sometimes created from spreadsheets with a little massaging ...
2
votes
3answers
26k views

Read CSV and perform SQL in SQL Server Management Studio

Is it possible in SQL Server Management Studio to read a CSV and perform an SQL update per line? Example data.csv "column1","column2,"column3" "column1","column2,"column3" "column1","column2,"...
2
votes
1answer
304 views

Looking for an efficient way to clone tens of millions of database records

A laboratory I work in is attempting to clone the open, biochemical database managed by PubChem found here. (Please don't ask why, I am not in charge of this project, I am simply tasked with the ...
2
votes
1answer
5k views

Dealing with empty strings while loading a table from a CSV

I am trying to load a CSV file into a table. The CSV file contains some empty strings for example: 1,2,,4 I want to load the empty field as null. One way I know is to use \N i.e: 1,2,\N,4 but it ...
2
votes
1answer
26k views

How to import a.csv file into a postgresql database using \copy?

I am trying to import a .csv file into a postgresql database. I am using pgadmin III for that on a win7 machine. My query looks like that: \COPY pop_grid(GRID_ID, POP_TOT, YEAR, METHD_CL, CNTR_CODE, ...
2
votes
3answers
6k views

how to import a huge csv file into Cassandra?

I am trying to import a csv file into Cassandra which is very long. These are food products: ingredients, nutrition, labels... It comes from Open Food Data. List information on food products: ...
2
votes
1answer
2k views

How to safely populate pg_notify payload?

Data makes its way into my application with prepared statements and stored procedures, so I'm not too worried about injection. When a new user signs up, I want to send an activation email. Presently I'...
2
votes
1answer
1k views

Syntax Error - OUTFILE with Timestamp

I have managed to come up wit the below script for exporting some files. This needs to be done on a number of datasets, and is done daily. The problem I am having is the I am getting the following ...
2
votes
2answers
902 views

Importing from CSV file, is crashing mysql server

I am facing a problem with importing large data into single mysql table. Basically I assume it is happening because limited machine resources but, I am having troubles to proof it. this is my CSV ...
2
votes
3answers
1k views

Best DB structure for large CSVs?

I have previously posted on stackoverflow, however, I have been directed here. I'm getting to the stage where I need to analyse lots of flat CSVs: Read-in around 500GB of CSVs (e.g. daily data split ...
2
votes
2answers
16k views

MySQL LOAD DATA INFILE Error Code 1366. Incorrect integer value: ' '

I've been trying to upload a CSV table into my remote server database but I find an error. I use MySQL in a RedHat Linux distrib. My mycsv.csv file looks like this: aa,ProductDescription,country,...
2
votes
1answer
2k views

Import CSV file - dynamic path

I am trying to import a csv file dynamically. This works: BULK INSERT #CSV FROM 'C:\Users\MyName\Desktop\Players.csv' WITH ( FIELDTERMINATOR = ',', ROWTERMINATOR = '\n' ) I would like to ...
2
votes
1answer
3k views

Bulk loading to partitioned table

I am designing a database application which will be using PostgreSQL 9.3. My question is regarding bulk loading a CSV to a partitioned table. I have done a lot of research, but I'm still unsure of ...
2
votes
2answers
1k views

Can't import flat file into SqlServer

I've got this tab-delimited flat file with fifty columns and two million rows. I am trying to import it into Sql Server 2017 Express. (I first tried working with it in Excel... hopeless.) I see two ...
2
votes
1answer
750 views

Wildcard for meta-command in postgresql

I am searching if there is a wildcard assigned to the \ meta-commandsSee description - meta-commands in Postgresql. for example.. postgresql=> \copy scheme.table FROM 'D:\path\to\%.csv' DELIMITER ...
2
votes
1answer
1k views

Postgres \copy from CSV: how to format range variables?

Consider a Postgres table containing a range variable: CREATE TABLE my_table (t_range tsrange, field1 text); If I want to manually insert a row via psql, I can run: INSERT INTO my_table (t_range, ...
2
votes
2answers
4k views

Importing CSV's into MySQL(5.6) tables leaves date fields as 0000-00-00

During the import from a CSV file created from an MS Access table into a MySQL table it converts all date fields in the csv into 0000-00-00. I use the following Load Data command for the import. ...
2
votes
1answer
2k views

Compact Spool file from Sqlplus

I have scripts that I run in Oracle SQL Developer and then export the result to my pc as a csv. I now started using sqlplus with a .bat to run my scripts and spool the result to my pc. But I noticed ...
2
votes
1answer
7k views

Error message when importing data into SQL (Import & Export Data Wizard)

I am trying to use the Import & Export Data Wizard to import the contents of a CSV file into a table in MSSQL. I have no issues if I import all columns as strings however, I need one column to be ...
2
votes
1answer
273 views

Powershell Output Help

I have wriiten the following SQL script to execute using powershell. cls foreach ($svr in get-content "demo.txt") { $con = "server=MC1-PQ10X.RF.LILLY.COM\SQL01;database=...
2
votes
1answer
2k views

Postgresql csv to db translate foreign keys

I'm trying to import data from a csv to a postgresql database. the db table looks like this: CREATE TABLE texts( text_id serial PRIMARY KEY, topic_id integer REFERENCES topics, format_id integer ...
2
votes
1answer
2k views

Load csv into MySQL database programmatically

I know about LOAD DATA INFILE command -- however, this requires that we already have the table with appropriate columns in place. When I have csv with hundreds of columns, it is not feasible to type ...
2
votes
1answer
148 views

Handling variable fields in a CSV file with JSON?

Im trying to figure out the engineering to get a CSV file into PostgreSQL. Requirements: The CSV has X number of predetermined required fields The CSV file has Y number of dynamic fields Example: ...
2
votes
1answer
519 views

postgresql - Importing CSV Data with Custom Type

I'm mostly a newcomer to databases and am building a postgres database of student achievement in a series of courses. My database uses a type called "class" that consists of the course, the academic ...
2
votes
1answer
5k views

MySQL: Load csv file to mysql

I'm having problems when loading a csv file into MySQL 5.7 with MySQL Workbench. Example Data ESH13 1/2/2013 10:04:35 1451 1 ESH13 1/2/2013 10:04:35 1451.25 2 ESH13 1/2/2013 ...
2
votes
2answers
1k views

Importing CSV file using SQL Server Import and Export Wizard (SQL Server Express)

So I am having a heck of a time importing a CSV file. The ultimate goal is to create a repeatable process to import the file. The only way I've been able to import the file is in a roundabout way ...
2
votes
3answers
4k views

Repopulating Oracle tables using data from Excel/CSV file

I'm developing a Java EE application that utilizes Hibernate/JPA. Some of the data I want to use comes from another Business unit's spreadsheet that I need to periodically (re)load into 5 Oracle ...
2
votes
1answer
525 views

Apply in a CSV file to multiple databases

Recently, I have been working with a client whom has asked us to provide their dataset for use in a mobile application we are making for them. The requirements for this project include the ability to ...
2
votes
1answer
6k views

MySQL import csv only gets half the lines

I use the following command to read in a csv: LOAD DATA INFILE '/Users/Tyler/Desktop/players_escaped.txt' INTO TABLE players FIELDS TERMINATED BY ',' ENCLOSED BY '^' LINES TERMINATED BY '\n'; The ...
2
votes
0answers
47 views

How To Query Databases in Subject-Predicate Format?

I have csv data in this format: id fieldname fieldvalue -- --------- ---------- 1 fieldA A1 1 fieldB B1 2 fieldA A2 2 fieldA B2 If I upload this to MySQL, how can I ...
2
votes
1answer
107 views

How do I bulk load data and still provide row-by-row feedback?

We want to allow our users to import CSVs containing many thousands of records to "pre-load" a particular table in our system. It's not strictly a one-time operation, it may be performed multiple ...
2
votes
1answer
153 views

PostgreSQL: simple equality query returning no results for indexed columns

I have a very large table (around 140M rows) with five columns: CREATE TABLE IF NOT EXISTS citations ( citing_id text, citing_issn text, cited_id text, cited_issn text, year int ); ...
2
votes
1answer
791 views

for Insert multiple rows in one single insert MySQL

I have done bulk-insertion in MySQL using multiple row insertions method by using python script. My testing csv file consists of 100,000 lines only (real one consists of millions lines). so I perform ...
2
votes
0answers
451 views

pgAdmin export line overflow on large field

Background: I have a database of significant geographical locations downloaded from GeoNames open database. The create script for the table is as follows CREATE TABLE populargeonameswiki ( geonameid ...
2
votes
0answers
7k views

sqlldr error - Invalid number

I am trying to load the following csv file into a table: TIMESTAMP ; SESSIONID;PROXY_SESSIONID; ACTION;USERNAME ;USERHOST ;OS_USERNAME ;CLIENT_ID ...
2
votes
0answers
2k views

CSV File Into MYSQL Merging - LOAD DATA on Duplicate Key?

Hey Dba Stack Exchange, My question to you is with regards to mysql. Right now I have a medium to small sized csv with 25k lines I need imported into mysql for a client. Problem is, their is only ...
1
vote
1answer
1k views

Fast way to import 2GB csv file into SQL Server 2008

I am new to database,I am using SSMS 2008 R2 to import 2GB of data from a .csv file. It is taking around 115mins to import. Can anybody let me know any better way to perform this import.
1
vote
1answer
104 views

Why does my database structure and SELECT operations generate NULLs?

I'm not too experienced with database problems, so please excuse the wall of text, I believe the context is important to understand the problem and goal. I receive a CSV file from another system ...
1
vote
2answers
33 views

How to create a csv with SQLCMD which can be imported into another database system

I may be doing this all the wrong way so please point me in the right direction if I am missing something. I have an Ubuntu server with SQLCMD installed and a Windows Server with a MSSQL database. I ...
1
vote
1answer
34 views

Copy large PostgreSQL table to multiple csv files with limited row number

Is there a way, using a sql function or an already existing psql function, to copy a very large table into multiple csv files and specify the maximum number of rows allowed in the output files ? For ...
1
vote
1answer
446 views

exporting a sql view to a csv file

Whilst I can just about script to pull data via queries, I'm finding it very difficult to figure out how to export the results of a query to a .csv file and save in a network storage location (novice ...
1
vote
2answers
185 views

How to split a string on every 2nd delimiter?

I have curves consisting of X/Y points (an array?) stored in a string with the same delimiter. There is 1 curve per row. Below is a simple example: 0,0,1,1,2,1.9,2.9,2.8,3.6,3.5 I want to extract ...
1
vote
1answer
2k views

Behavior of PostgreSQL COPY FROM

Let me preface this question by stating that my experience with DBA is limited to an introductory undergraduate university course, and that I'm just getting started with PostgreSQL. I would like to ...