Questions tagged [csv]
Questions on how to import into, extract data from, or exchange data between databases in CSV format (plain text using commas to separate values).
260
questions
0
votes
0answers
17 views
Restore a MySql database from CSV files [closed]
I've been doing investigations in ways to backup a database contents besides disk snapshots and good old mysqldump.
I've found tools to export database contents to parquet files, which if I understand ...
0
votes
0answers
12 views
When setting up an CSV Foreign Data Wrapper for Postgres what's the best way of dealing with \N
I am trying to set up a foreign data wrapper for CSV data dumped from a MySQL database. I have created the FDW server and set up the table but whenever I try and run any queries it raises an exeption ...
0
votes
0answers
15 views
Azure Database for MySQL - mysqlimport command
So I'm trying to accomplish the goal of performing a mysqlimport on a Azure MySQL 8 database and I get the following error:
mysqlimport: Error: 1227 Access denied; you need (at least one of) the SUPER,...
1
vote
2answers
63 views
Database Mail has incorrect encoding of CSV attachment
I have a procedure that generates a resultset, which I attach as a CSV file attachment in sp_sendmail. However, the resulting file is encoded UCS-2 LE BOM (according to notepad++). This causes the ...
0
votes
1answer
28 views
importing csv file with ragged right format and " text qualifier
I'm importing csv files in ssis that are in a ragged right format, I've been able to create the columns perfectly fine but for some of the data rows it has included a " text qualifier into the ...
-1
votes
1answer
12 views
Can I use a string as an end of line delimiter when importing a .csv to MySQL?
Very basic question but I'm struggling with it. When importing a .csv file to MySQL, is it possible to use any character or string as an end of line delimiter?
If so is it possible from the admin ...
0
votes
1answer
18 views
Export mutliple query output into one excel sheet using Redshift and SQL Developer
I am connected to Redshift from SQL Developer 19.2.1.
I have a requirement to export multiple query output into a single excel sheet, one query output should be in one Tab. Please advise on how to ...
0
votes
1answer
86 views
(PostgreSQL 11) Reading in an array of JSON objects (or composite types) from a CSV file into Postgres
So I am trying to import rows into a table using a CSV file, but have been running into problems I haven't been able to solve. I would like one column of a row to be an array of JSON objects.
How I ...
1
vote
2answers
22 views
BULK INSERT does not fail when file contains commas instead of semicolons when FIRSTROW > 1
I encountered an issue where I received a CSV file that supposed to be delimited by semicolons (;) but was delimited by commas (,).
Bulk insert called by sqlcmd did not fail but did not insert either. ...
1
vote
3answers
151 views
Japanese characters in VARCHAR column using Japanese_Unicode_CI_AS
I am loading a table with the Japanese names from CSV file that are changing into question marks into SQL Table. The table that is storing these values is a varchar column. I understand that the ...
0
votes
0answers
34 views
Upload CSV data on AWS RDS PostgreSQL
I've built a basic webpage to push data to the PostgreSQL database on AWS RDS from a CSV file. The issue is the docs they have provided use CLI to do that. If I use direct SQL queries using \copy ...
0
votes
0answers
33 views
mysqlimport and avoiding table lock
I reviewed the question & answer here (mysqlimport locks all tables system wide) but I seem to still be stuck on this.
I am standing up a new master/slave pair of 5.7 servers, and have already ...
-2
votes
2answers
39 views
How to find out WHERE a PG error/warning/log happened?
I have PostgreSQL CSV-log its "errors" according to this structure: https://www.postgresql.org/docs/12/runtime-config-logging.html#RUNTIME-CONFIG-LOGGING-CSVLOG
It doesn't include the file ...
1
vote
1answer
177 views
SQL Server BULK IMPORT Truncation error
I am trying to import a simple CSV file into a table on SQL Server 2017. The CSV file and table match each other in number of columns and data types. This is an extract of the CSV file:
UID,...
-1
votes
1answer
33 views
How do I log the actual script's name in csvlog? [duplicate]
I have PostgreSQL CSV-log its errors according to this structure: https://www.postgresql.org/docs/12/runtime-config-logging.html#RUNTIME-CONFIG-LOGGING-CSVLOG
Sadly, from what I can tell, it neither ...
0
votes
3answers
79 views
Import daily csv with varying available headers
I receive a csv daily from an external company capturing feedback for us.
Unhelpfully the csv headings vary. There could be any combination of headers.
For example:
day1: heading1, heading2, heading3, ...
0
votes
0answers
92 views
PostgreSQL 10 - create partitions on column that needs to be calculated after importing a csv
I am in the situation where I need to create a table which also has a time column.
This table is filled with data taken from a CSV, but the csv has no time column.
Only after importing the data inside ...
0
votes
1answer
792 views
ORA-06502: 'character string buffer too small' when writing to a csv
I have some tables that i want to export. I'm using this package (I called it "CSV"): https://www.williamrobertson.net/documents/refcursor-to-csv.shtml
I was able to write to CSV files ...
0
votes
1answer
39 views
attach csv file on email sent from postgres
I have the following code to send emails and wanted to see if it's possible to attach a csv file?
CREATE OR REPLACE FUNCTION public.send_email_html(text, text, text, text)
RETURNS integer
LANGUAGE ...
2
votes
2answers
204 views
using SQLCMD to export/extract data withot trailing spaces
I am trying to extract data from some of my tables using sqlcmd utility as SSMS does not generate appropriate csv-s for me.
My issue: all csv extract includes trailing spaces according to table ...
1
vote
1answer
119 views
Copy csv file into different schema [closed]
I need to import data from a csv file which from database a schema a into another database b schema b.
I used the pgAdmin running the export and when I import the data using the same tool pgAdmin and/...
1
vote
0answers
225 views
Export and split QUERY results in multiple files
I would like to know if it's possible to export and split query results in multiple files. Let's assume I have a query returning 100 results and I want to save the results in 5 different csv files (20 ...
1
vote
0answers
2k views
Persistent problems importing CSV files via pgAdmin 4 to PostgreSQL 12 on Mac OS - Failing with Exit Code -6
I've been trying for several days to import CSV files into a PostgreSQL 12 database hosted on Mac OS 10.15.5, using the Import/Export tool in pgAdmin 4. I am consistently getting an error message ...
0
votes
0answers
385 views
Attempted to import .csv into mySQL using workbench, but only imports 7 rows?
To start, I'm still a relative SQL noob so this question could have an obvious answer that I'm just missing. I'm trying to start a big practice project in my spare time using a dataset which has ~...
0
votes
1answer
330 views
MS SQL Server: Import Flat File with auto-increment ID
So I've downloaded a sample database. It originally didn't have an ID column so I inserted one in through Excel. Anyways, I went through DatabaseName > Tasks > Import Flat File and the usual ...
0
votes
2answers
74 views
RDBMS: Proper way to store data - comma separated variables or in different fields or tables?
Fundamental question that I cannot seem to find an answer on. I have a database that stores line items from receipts along with the username and receipt number.
Right now the lineItem column is just ...
0
votes
1answer
5k views
PostgreSQL: date/time field value out of range
Software:
Linux cjz-eshop1-p 5.4.0-33-generic #37-Ubuntu SMP Thu May 21 12:53:59 UTC 2020 x86_64 x86_64 x86_64 GNU/Linux
psql (PostgreSQL) 12.3 (Ubuntu 12.3-1.pgdg18.04+1)
I am getting errors due to ...
0
votes
2answers
1k views
How to import multiple CSV files to a postgres table using pgadmin or other method?
I'm looking to import a folder of over 100+ csv files into one table on PostgreSQL using pgadmin. I have the following script but is there a way to change the file path to all csvs?
COPY schema.table ...
1
vote
1answer
84 views
SQL (Redshift) SELECT COUNT from CSV column
Hi exchange community,
Problem statement and details:
I am trying to query a column that contains CSV data to produce a count of instances of each value in the table. There are a total of 14 ...
0
votes
1answer
100 views
How can I monitor the import progress of a large CSV File in mysql?
I am trying to import an existing CSV file into my database and I want to see the progress. I usually use pipe viewer when I do my sql files but for csv files I cannot figure out how to do it.
I use ...
0
votes
1answer
646 views
MySQL Workbench: I need to update values in a table based on values from a csv
Newbie Problem, but I've tried creating a new table and importing the csv to that then updating, but I have been having trouble with that, any guidance would be greatly appreciated.
I have a csv file ...
0
votes
0answers
26 views
CommandLine for Exporting All MS SQL Tables to CSV
On my Mac, I have a docker container running an Ubuntu version of MS SQL. I have a DB running in it and I can access it via Azure Data Studio, and I can view each table individually and export it to ...
0
votes
1answer
117 views
How to Export MS SQL DB from Docker-Linux to Access format
I have a MS SQL database which is currently up and running on my Mac within a Docker container that is hosting the Ubuntu Linux version of MS SQL Server. I need to export the database to a format ...
0
votes
2answers
2k views
Why BULK INSERT insert data in random order?
I have a csv file with 350,000 rows. Would like to insert data to temporary table in same order as is in csv file. I am trying to BULK INSERT using:
BULK INSERT ##AllRows
FROM @FilePath
WITH
...
1
vote
0answers
24 views
How to migrate data from a large amount of csv files with time x space measurements to a database that needs to be available for analysis [closed]
I have made a simulation experiment to obtain time series measurements at different points along a power line. These measurements were done for different types of power lines in different test ...
0
votes
0answers
684 views
Import from csv file to SQL table - Double quotes on the very end
I was able to import a .CSV file into my sql table, here is my BULK INSERT script:
BULK INSERT TLS_Sysprocess
FROM 'D:\Dennis\DevAPPServers2.csv'
WITH (
FirstRow=2
,FIELDTERMINATOR = '","'
,...
0
votes
1answer
397 views
Postgres randomly fails silently on huge CSV import
With Postgres 10.12 on Windows 10, I am importing at 67 GB CSV file into a database using a simple COPY query:
COPY table_name FROM 'c:/temp/data.csv' WITH (FORMAT csv, HEADER);
The import fails ...
0
votes
1answer
46 views
CSV or DB is quicker to read several lines from a file with 1,5 billion lines
I have a program which would read several lines according to the indices of lines within a CSV file with 1,5 billion lines(using c fgets, or is there any tool in C faster than fgets?). Compare with ...
0
votes
1answer
34 views
How often is Postgres parsing a foreign table based on file_fdw?
I have a foreign table defined with file_fdw, which reads the data from a CSV file. Postgresql parses the CSV file and converts the date for example from YYYYMMDD to YYYY-MM-DD. This works fine.
My ...
0
votes
0answers
12 views
MySQL 8.0 appending unknown character to column
I'm inserting a CSV files into MySQL 8.0 using Python 3.7:
bulk_insert_sql = "LOAD DATA LOCAL INFILE filename.csv INTO TABLE table_name FIELDS TERMINATED BY '|' LINES TERMINATED BY '\n' IGNORE 1 ...
0
votes
1answer
700 views
Mysql output to csv putting column names on the bottom
I'm trying to export the results of the query listed below to csv and have the column names in the first row. However, when I run the code and open up the csv the header is listed as the last row. I ...
0
votes
0answers
70 views
LOAD DATA INFILE NULL are enclosed not loaded
I have a CSV file that I exported from Oracle database. All the integer columns are not enclosed, but all the string columns are enclosed with "
I'm using MySQL 8. So I used the following command ...
0
votes
2answers
46 views
Ram & Concurrency: MySQL InnoDB LOAD INLINE FILE on 50 30GB CSV files
As the title suggests I'm moving around a lot of data. This is in an environment with a 24 core CPU and 2TB of M.2 storage.
My first question is: given I have 16GB of ram, does that mean it's ...
1
vote
1answer
2k views
Copy large PostgreSQL table to multiple csv files with limited row number
Is there a way, using a sql function or an already existing psql function, to copy a very large table into multiple csv files and specify the maximum number of rows allowed in the output files ?
For ...
0
votes
2answers
133 views
How to automate the detection of the column type of a csv file for a table creation script?
I am new to pgAdmin and Azure database. I have a huge csv file with around 220 columns and I want to create a table out of it in pgAdmin4 to push it to Azure.
However I don't know how to detect ...
1
vote
2answers
160 views
How to create a csv with SQLCMD which can be imported into another database system
I may be doing this all the wrong way so please point me in the right direction if I am missing something.
I have an Ubuntu server with SQLCMD installed and a Windows Server with a MSSQL database.
I ...
0
votes
1answer
371 views
How to add multiple query results into one csv file?
By the help of the members of this community I have created a script which shows recent T-log backup for each database in the server. Instead of sp_MSforeachdb I used a customized script written by ...
0
votes
0answers
48 views
Import folder with Multiple CSV's into SSMS w/ out SSIS
I am wondering what's the best and most effective way to import csv's stored in a local folder into DB w/ in SSMS.
Folder path- C:\Users\ngutierrez\Desktop\Open_Air_Tables
Folder contains about 30 ...
1
vote
1answer
2k views
exporting a sql view to a csv file
Whilst I can just about script to pull data via queries, I'm finding it very difficult to figure out how to export the results of a query to a .csv file and save in a network storage location (novice ...
0
votes
0answers
145 views
Errors with trying to run a stored procedure with local file output
I'm trying to run a stored procedure that I'm passing some variables to.
The output is being writted to a CSV file on a local file share.
The CSV file gets created, but I'm hitting some errors ...
