Questions tagged [csv]

Questions on how to import into, extract data from, or exchange data between databases in CSV format (plain text using commas to separate values).

Filter by
Sorted by
Tagged with
3
votes
1answer
699 views

T-SQL: CSV->table pipeline with custom parsed numeric data, lookup values

Problem Wish to insert thousands of records into a table [Child] from a CSV file (and do this from tens of files into the same table). However, I want to A. substitute a lookup value into the table (...
3
votes
1answer
209 views

Dynamic paths for postgres file_fdw (csv)

I've got a postgres database which relies on the use of foreign tables to support the loading of large data sets based on user-provided (system sanitized) tables. I'd use COPY, but the tables loaded ...
6
votes
2answers
1k views

Very large SQL Server 2016 result sets (over 75 GB) to a CSV file?

What is the best way to get very large SQL Server 2016 result sets (over 75 GB) to a CSV file? The engineers need this output to look for correlations. The bcp route for a 73.5 GB file filled up ...
2
votes
1answer
14k views

Import Data from 48 GB csv File to SQL Server

I am importing huge data file of size around 48 gb using SQL Server default Import Tool. It keeps on executing fine for app. 13000000 Row insertions but after it the Task Fails with following error. ...
1
vote
1answer
482 views

select distinct on csv data

A table has CSV data and I want to do a SELECT DISTINCT ID on that table, how should I proceed? +-----------+ | id | +-----------+ | 111,1487 | +-----------+ | 462 | +-----------+ | ...
1
vote
1answer
102 views

Why does my database structure and SELECT operations generate NULLs?

I'm not too experienced with database problems, so please excuse the wall of text, I believe the context is important to understand the problem and goal. I receive a CSV file from another system ...
0
votes
1answer
64 views

How to fetch records from table which has csv

Please guide me about following case. I need to fetch records from a table which has a column like shown in the picture. I need to fetch records which has 065 and AMT campaigns only. I tried: SELECT ...
1
vote
0answers
396 views

Which method gives better insert performance in mysql db table: CSV import or insert statement

I have combine data from two tables which results in 6727224 rows. I want to insert this data into another mysql db table. Which method should be very fast to insert this number of records. Insert ...
0
votes
1answer
499 views

SQL server Import CSV file - 1'st row is also added as s column name

I'm trying to import a CSV file using import export wizard, Im facing an issue. The first row is also added as a column name. Import wizard: The highlighted area is my 1'st row.
2
votes
1answer
5k views

MySQL: Load csv file to mysql

I'm having problems when loading a csv file into MySQL 5.7 with MySQL Workbench. Example Data ESH13 1/2/2013 10:04:35 1451 1 ESH13 1/2/2013 10:04:35 1451.25 2 ESH13 1/2/2013 ...
2
votes
1answer
2k views

How to safely populate pg_notify payload?

Data makes its way into my application with prepared statements and stored procedures, so I'm not too worried about injection. When a new user signs up, I want to send an activation email. Presently I'...
0
votes
2answers
82 views

Specify Import Row Range for SQL Server Task Wizard

I would like to import a csv file into an empty table in SQL Server Developer 2016, using Task Wizard. I know with BULK INSERT there are options to specify firstrow and lastrow; does Import Wizard ...
0
votes
2answers
618 views

MySQL Import & Replace via CSV

I have a MySQL database table named items with 10,000+ records. Of these records, I would like to export only records where item_isbn IS NULL. SELECT * FROM Items WHERE item_isbn IS NULL produces a ...
0
votes
2answers
87 views

Using Structured Filetypes as Databases

In having a debate with a coworker it was posed that a flat file, in this case a CSV, could be considered a database. This would include XML and JSON as well. I'm interested in differing views on this....
0
votes
1answer
4k views

How to export data in Oracle database into CSV file with UTF-8 format?

I have an CSV file with WE8ISO8859P1 encoding generated from Oracle Stored Procedure. Is there any script line to define the output of CSV file in UTF-8 encoded format? CREATE OR REPLACE PROCEDURE ...
1
vote
1answer
1k views

Fast way to import 2GB csv file into SQL Server 2008

I am new to database,I am using SSMS 2008 R2 to import 2GB of data from a .csv file. It is taking around 115mins to import. Can anybody let me know any better way to perform this import.
0
votes
2answers
2k views

How to SELECT with two CSV files/… on PostgreSQL? [closed]

I have two CSV files which you can insert into the following PostgreSQL schema. However, I cannot \COPY the data into the tables. Pseudocode Let data_id=1 Show name and event_value from data.csv and ...
3
votes
2answers
74 views

Generate CSV Email For Each UserID In Table

Good day all, I have a unique requirement where I need to generate a .csv file containing data for each userID. The result set that I have returned from my select statement is userID ...
5
votes
1answer
4k views

Order the result of a comma separated string generated with STUFF

Given a schema like this: CREATE TABLE Foo ( Id int PRIMARY KEY, Position int NOT NULL, Title varchar(10) NOT NULL ); INSERT INTO Foo VALUES (1, 3, 'Title3'), (2, 10, 'Title10'), (3, 1, ...
6
votes
2answers
5k views

Select a CSV string as multiple columns

I'm using SQL Server 2014 and I have a table with one column containing a CSV string: 110,200,310,130,null The output from the table looks like this: I want to select the second column as multiple ...
0
votes
2answers
439 views

How do I import updates to unique data in MySQL

I have a reasonably large MySQL database that is an import of CSV values from a public data repository. It has come to my attention lately that they occasionally update (ie. correct) data, usually ...
3
votes
1answer
729 views

bcp error: date invalid date format

I'm a bcp newbie. I want to insert a large file. I'm having a hard time uploading datetime format. Table SQL: "Create Table dbo.my_test ( " + \ "transaction_id BIGINT ," + \ "transaction_no ...
1
vote
1answer
3k views

Failed import of date using COPY

I'm trying to import a CSV file into a table in Postgres, and it's bombing out on the first date field. The dates are in the format dd/mm/yyyy and there isn't a date in the field in question in the ...
4
votes
3answers
2k views

import 100 csv files into postgresql

I have a 100 csv files named sequentially 1,2,3....100.csv I copy the csv files with this script: COPY location FROM 'C:\Program Files\PostgreSQL\9.5\data\´1.csv' DELIMITER ','; I would to ...
1
vote
1answer
875 views

Best data upload formats

Most of our automatically generated files contain integers, decimals, date, time, and sometimes varchar. This is instrumentation data. Since we will be uploading those files directly to a SQL Server ...
2
votes
1answer
3k views

Oracle SQL Developer: Set the <null> value when exporting to CSV

When exporting to CSV in Oracle SQL Developer, null values are replaced with "". I want them to be exported as the string null. Is this configurable anywhere? Example of what is wished: "STARTDATE"...
3
votes
2answers
3k views

How can I output more than 256 characters to a file?

I am using the following command to write the output from a stored procedure to a file for further processing: DECLARE @shellCommand VARCHAR(2000) SET @shellCommand = 'SQLCMD -S ' + @@SERVERNAME + ' -...
0
votes
1answer
582 views

Dump MySQL table to csv file (utf8mb4_unicode_ci)

I have MySQL table with columns (character-set: utf8, collation: utf8mb4_unicode_ci) to be exported to a .csv file. mysqldump -u *** -p -t -T/tmp/ dbname tablename --default-character-set=utf8mb4 ...
1
vote
4answers
2k views

Bulk table creation and import of data

I have a around 400 CSV files which have been exported from an old Paradox database, with each CSV representing a table within the old database. Each CSV has a different number of columns and ...
2
votes
1answer
765 views

for Insert multiple rows in one single insert MySQL

I have done bulk-insertion in MySQL using multiple row insertions method by using python script. My testing csv file consists of 100,000 lines only (real one consists of millions lines). so I perform ...
0
votes
1answer
697 views

Looping all csv files in local folder for data insert in MySQL db ( Both are local

Below is showing error "for f in 'C:\Users\Documents\TEMP*.csv DO LOAD DATA INFILE '"$f"' INTO TABLE TEMP FIELDS TERMINATED BY ',' ENCLOSED BY '"' LINES TERMINATED BY '\r\n' IGNORE 1 LINES" DONE" ...
5
votes
3answers
5k views

How do I disable escaping when using COPY FROM in PostgreSQL?

I have a large tab delimited file that I want to read into a table in PostgreSQL 9.5. It contains double-quotes and backslashes that I want to treat as regular characters. I think COPY FROM is the ...
2
votes
1answer
706 views

Wildcard for meta-command in postgresql

I am searching if there is a wildcard assigned to the \ meta-commandsSee description - meta-commands in Postgresql. for example.. postgresql=> \copy scheme.table FROM 'D:\path\to\%.csv' DELIMITER ...
5
votes
3answers
9k views

Import in postgres json data in a csv file

I'm trying to import into postgres a csv file containing the data for a table. One of the column of the table has jsonb type. One line of my csv file contains something like 1,{"a":"b"} Suppose the ...
0
votes
1answer
754 views

Invalid directory path error while connecting to database

I am using oracle 11g and using sqldeveloper to run/edit queries. My table is: ID FIRST_NAME LAST_NAME MAJOR CURRENT_CREDITS 10008 A Dd Computer ...
3
votes
1answer
3k views

Why doesn't postgresql COPY command work with files in mapped drives

I have a problem copying CSV files from a network shared drive into postgreSQL, at present the fix is to copy the files to a local drive first, but it's another process in a chain which is becoming ...
1
vote
0answers
458 views

Import Data From csv Into Postgresql WITHOUT COPY

I am writing a function for a python environment which needs to import data from a csv file into an existing table. The scope in which the function can execute does not allow for a db SUPERUSER to run ...
1
vote
0answers
114 views

Postgres 9.4 importing csv file causes connection abort

I have very simple database which i would like to import data from a csv file, so far i tested with a file with a few rows, and it worked. So tried to upload the real data to the database, which ...
-1
votes
1answer
3k views

insert by loop in sql

I am very new to SQL SCRIPT and I am trying to insert elements in a loop.I initially created a table "student" with two columns "id","name". Now I have a CSV file of 1,"Sam" 2,"Ranger" 3,"Hardy" 4,...
0
votes
1answer
453 views

SSIS - XML with row delimiter to a single column of a SQL table

I'm trying to upload data from a CSV file into a SQL server table through SSIS package (Flat File source and OLEDB destination). This CSV has comma as column delimiter and {CR}{LF} as row delimiter. ...
0
votes
2answers
806 views

convert csv to insert sql statements

I have a CSV file and I want to convert it into a SQL INSERT statements.But I could only find online converters but I would like to know can I create it manually ?
1
vote
1answer
313 views

how to convert my PostgreSQL database table into CSV? [closed]

i need to convert my PostgreSQL table into csv, how to do this? i have visited many sites they suggest the PostgreSQL command to convert the table into CSV but i always get the error message. the ...
5
votes
3answers
819 views

Export to format compatible with SQL Server

I need to provide data in a format that could be imported into a Microsoft SQL Server database; however, I do not have Windows or access to a SQL Server instance. The data is currently in several CSV ...
2
votes
1answer
1k views

Syntax Error - OUTFILE with Timestamp

I have managed to come up wit the below script for exporting some files. This needs to be done on a number of datasets, and is done daily. The problem I am having is the I am getting the following ...
0
votes
1answer
525 views

How can I (re-)create the Entity Relationship Diagram (ERD) of an existing hidden Oracle database?

More precisely, I wan't to re-create the database model / schematics / the Entity Relationship Diagram (ERD) showing all the links between the tables of this existing hidden Oracle Database. In my ...
1
vote
2answers
774 views

Export SSRS report to CSV

I would like to export an SSRS report to a CSV file. Sounds simple, but I do not know how to add double quotes to the column headers in the exported CSV file. For example: "UserName"|"UserAdd" "Mike"|...
4
votes
3answers
8k views

What's the most efficient way to load a csv file into SQL Server 2016?

Alright, so this question seems simple up-front. Well, there's a bit more to it than just loading the file. The entire story goes like this: For some reason, our client sends us what can only be ...
0
votes
1answer
73 views

How to store data having a fixed number of columns and a variable number of rows

I have posted this question in the stack overflow site and a more experienced user than me suggested I post it here. I am quite new at collecting data and am collecting a new dataset using R from ...
0
votes
1answer
56 views

Create a MySQL Database using a MS SQL database dump (.CSV)

I have a dump of a MS SQL database in the form of CSV files for every table in the database. There are 679 .csv files each for every table. The highest size of the table extract is 9 GB and the lowest ...
2
votes
3answers
23k views

Read CSV and perform SQL in SQL Server Management Studio

Is it possible in SQL Server Management Studio to read a CSV and perform an SQL update per line? Example data.csv "column1","column2,"column3" "column1","column2,"column3" "column1","column2,"...