Skip to main content

Questions tagged [csv]

Questions on how to import into, extract data from, or exchange data between databases in CSV format (plain text using commas to separate values). Rows are typically denoted by newline characters.

Filter by
Sorted by
Tagged with
-3 votes
0 answers
12 views

no COPY permission with CSV

i read all post with denied permission problem. But I need a COPY in script (python psycopg2), I can't use psql \copy I'm on MX-LINUX, postgresql 15. My role is superuser, i'm DBA of my database, I ...
pasqal's user avatar
  • 95
1 vote
1 answer
424 views

Can SSIS read the results of a stored procedure and write them to CSV, without populating a table?

My current workflow is as follows: A 'Execute SQL Task' task runs a stored procedure that populates a table. After this, a 'Dataflow Task' task does the following: 2A. Read every row from this table ...
J. Mini's user avatar
  • 533
1 vote
2 answers
108 views

Mariadb update column during load data of CSV

I am loading .csv files into Mariadb 10.6 using LOAD DATA LOCAL INFILE. There is a DATE column called posted_on in the inbound data, and I need store the month name in another field monther, ideally ...
br8k's user avatar
  • 115
0 votes
1 answer
218 views

batch script to get all the tables in postgres database as csv

I have a remote postgresql database with a lot of schemas. I want to dump the tables that is specific schema as .CSV file for each table. I'm aware of the COPY command but I'm not sure how to write a ...
new_developer's user avatar
0 votes
1 answer
105 views

How to export db file from SQLLite

I have uploaded my csv file to https://sqliteonline.com/ and it is currently available as a table. I need to export this data to .db file to use it elsewhere. I tried exporting the table, but I only ...
User 19826's user avatar
0 votes
0 answers
94 views

Column ___ of relation ___ does not exist in csv migration -- is Liquibase ignoring the schema I configured?

edit: I fought this too long and went with SQL insert statements instead, since I was generating the CSV from a JS script it was easy to translate over. I think there's a bug in this loadData command. ...
jcollum's user avatar
  • 207
0 votes
2 answers
382 views

How to Import 21k Comma Delimited Text Files having a few columns with quote delimited comma separated values into SQL Server

I inherited a broken process, need to fix it and I'm hoping you, my fellow DBAs, may have a few good solutions. There's a Windows server folder containing 21,663 CSV files that need to be imported ...
JasonVL's user avatar
0 votes
2 answers
3k views

msg: Cannot execute SQL 'COPY "my_table" FROM '/path/big_file.csv': ERROR: Could not extend file "base/A/B": only C bytes of D written in block E

I am running an ansible playbook. The machine running ansible is localhost. The playbook breaks during a task which loads data from a big csv (1.2 GB) into a postgres database table ( module community....
Tms91's user avatar
  • 101
0 votes
0 answers
219 views

I cannot write information to a filesystem file from my newly installed MS SQL Server

Hello and thank you for your help. I must write non-tabular information to a file. I have tried a few methods that invoke various stored procedures. The following SQL statement tests this and I ...
Craig McWilliams's user avatar
0 votes
1 answer
818 views

Importing CSV File into MySQL gives UnicodeDecodeError

I'm trying to import a CSV file into the insert tab of a MySQL database model. Importing the file gives the following error: Error during data import: UnicodeDecodeError("'ascii' codec can't ...
SlowlySwift's user avatar
-1 votes
1 answer
224 views

How to load data into a table from a file and set other column(s) too?

I have a table with multiple columns. And there's also a text file which contains data, one value per line: aaa bbb ccc ddd How would I load a file into a file into a certain column as well as set ...
Camila326's user avatar
0 votes
0 answers
38 views

Load One Field CSV into Table

I have a CSV file which contains 1 filed. I want to import this filed values into a MariaDB table using the LOAD DATA INFILE command. This table has 9 columns with existing data (about 1 000 000 rows)....
Hisab's user avatar
  • 1
0 votes
2 answers
475 views

Loading multiple CSV files into mariadb table using LOAD DATA INFILE and shell script

I'm attempting to load CSV data into a mariadb table. MariaDB is running in a docker container and the CSV files have already been copied in the container as well. I'm already able to successfully ...
Roshan Upreti's user avatar
1 vote
1 answer
613 views

How to import CSV files into MariaDB in a secure and performant way?

We have a PHP based web application that allows the upload of CSV files. These CSV files with up to 2.5 million rows are to be imported into a table in a MariaDB. How can we make this import secure ...
Ste2play's user avatar
0 votes
1 answer
6k views

ERROR: extra data after last expected column when inserting data into PostgreSQL from a CSV file

I'm inserting data into a PostgreSQL (15) database from a quite large CSV file containing image metadata but I'm facing the error at some point: ERROR: extra data after last expected column CONTEXT: ...
s.k's user avatar
  • 372
0 votes
0 answers
1k views

Postgresql import csv failing on quoted \r\n

This is my first question here and I'm new to Postgres as well. I'm trying to import a csv file into an existing table that has an identical structure. Using the PgAdmin4 Import/Export data... I get ...
JHerbst's user avatar
0 votes
2 answers
1k views

How can I use pg_dump to export a single table to CSV that can then be imported by Oracle SQL Loader?

Thanks in advance for any help on this. Problem Summary: I'm looking for the most efficient way possible to export a single table from Postgres/Greenplum for a large number of records (100M+) so that ...
zigmoo's user avatar
  • 9
7 votes
2 answers
2k views

Advantage of "LOAD DATA ... REPLACE INTO TABLE" over "UPDATE table_name SET"

I inherited a system in which all updates (even to a single row/record) to MySQL tables are not done using UPDATE table_name SET. Rather, they are done by: Exporting the existing table to a CSV (text)...
WebViewer's user avatar
  • 173
2 votes
1 answer
276 views

Not able to wrap values in quotes when exporting CQL table to CSV

I am not able to copy data into a CSV file and have them wrapped with single quotes: e.g. Table: myks.mytable column1 column2 value1 value2 command used: copy myks.mytable (column1,column2) to '...
Tarik Haddad's user avatar
1 vote
1 answer
566 views

Flatten a JSONB file - What function/query

In a bit of a pickle at work - I have a Jsonb which I need to flatten a file that has similar structure: { "Manufacturers":[ { "manufacturer":"Tesla", ...
NasahT's user avatar
  • 25
1 vote
2 answers
3k views

Converting PostgreSQL dump file into CSV files

I have a PostgreSQL database running in a docker container. I used the below command to get a dump file of the Postgres database - "docker exec -t your-db-container pg_dumpall -c -U postgres > ...
sreejan patel's user avatar
1 vote
1 answer
89 views

Upgrading from excel to an actual database

Just started my first internship and I could use some guidance on approaching the project given to me. Simply put, all that needs to be done is to make a database and then transfer the data from a ...
John common's user avatar
0 votes
3 answers
64 views

MySql query taking long time while querying 5s resolution 4 years data

I am downloading MySql query. It is taking long time. Long solution I found is, download 2 months data at a time. I don't like to do it almost 25 times same thing. Is there a better way I can query ...
user avatar
0 votes
1 answer
273 views

Syntax Error in Boolean Null values

I have a csv that contains Boolean variables (yes,no,blanks). How can I recode blanks so that it'd integrate in the data type. Or fo i have to change data type for blanks to be readable?
Saransh's user avatar
0 votes
1 answer
4k views

Importing CSV with a json array of strings in one of the column

The columns of the CSV are of the format text (can include empty lines), date and a JSON array of strings (something like ['a', 'b', 'c']. I've been trying to copy that CSV to a PostgreSQL table (...
bqback's user avatar
  • 103
0 votes
1 answer
2k views

0 rows from the .csv (semi colon) are imported into MySQL Workbench

I am trying to import the .csv below (columns and 3 first records in txt in differenet blockquotes) into MySQL Workbench using the Table Data Import Wizard. I have tried many ways and realized that ...
transmod_kdtk's user avatar
0 votes
1 answer
110 views

Postgres improvements in conversion functions for bulk load

I regularly import data from httparchive.org. The data is a MySQL CSV export and I use pgloader, which handles the quirks of this export (\N for NULL), etc. I also need to do some additional ...
Charlie Clark's user avatar
1 vote
1 answer
1k views

SQL in Access to add leading 0 in front of zip codes

for a project I am working with a table of zip-codes that I imported from a *.csv file, exported form Excel. Those zipcodes all have 5 numbers and are formatted as text within my database. Some of ...
Titan's user avatar
  • 11
2 votes
1 answer
8k views

Date not being imported from csv in proper format in MySQL

I am new to MySQL, so kindly explain your answers in a simple way! I have a csv file which has 2 date columns. I want to import the csv into MySQL and I am using the following method: Create an empty ...
spectre's user avatar
  • 121
1 vote
1 answer
1k views

DB strategies to store data in CSV files?

I'm new to the Postgres community and would love to hear your recommendations. To give a bit more context, each file is associated with a unique zip code and address and contains 8,760 (365 days x 24 ...
drao96's user avatar
  • 13
1 vote
1 answer
6k views

unterminated CSV quoted field when to import csv data file into postgresql

Create a table : CREATE TABLE num ( id serial NOT NULL , adsh VARCHAR(20) NOT NULL, tag VARCHAR(256) NOT NULL, version VARCHAR(20) NOT NULL, coreg VARCHAR(256), ddate date NOT ...
showkey's user avatar
  • 334
0 votes
2 answers
501 views

Export colums containing (geo)JSON elements to CSV while keeping the JSON double quotes properly

I want to export using copy 5 columns of a table, 3 of them contain ID, text and timestamp values but 2 of them contain JSON and GeoJSON elements, e.g.: // Please, note the non-quoted string Value1 /...
s.k's user avatar
  • 372
0 votes
1 answer
1k views

How to save a CSV file that adds double quotes to special characters that enables easy importing?

When I was importing a CSV file to MySql Workbench that had special characters like comma, brackets, etc., few rows that contained data containing special characters were getting missed (, )) etc). ...
Gaurang Agrawal's user avatar
0 votes
2 answers
1k views

Sqlite3 import csv column ordering

I have a situation where I have CSV files with column names in the first row, which perfectly match the tables in my SQLite3 db, except they are in a different order. By default, with .import table1....
Max888's user avatar
  • 101
0 votes
0 answers
349 views

How to move/copy a CSV file saved in /var/lib/mysql-files without root on a shared hosting server

I'm working with a server that I'm helping someone with to extract data from their database. The database server is no a shared hosting plan as is very very slow to run queries. Query that takes 3 ...
TheRealChx101's user avatar
1 vote
1 answer
2k views

How do I import CSV UUID data into Postgres on the command line?

I'm using PostGres 10. I have this table > \d myapp_currencyprice; Table "public.myapp_currencyprice" Column | Type | Modifiers ----------------...
Dave's user avatar
  • 723
0 votes
0 answers
333 views

MariaDB veryslow Inserts Query/Import via Dbeaver

I have a MariaDB server that via DBeaver is getting very slows insert times. Tables with 10.000 lines only is getting imported in 2minutes The DB Manager is trying to import CSVs, some with only 80mb ...
André Bachtold's user avatar
0 votes
1 answer
575 views

Get data from 2 tables based on comma separated values Join

I am an average in database operation but I never deal with a database having values in comma separated manner. As I want the result from both table. I have one table "Form table" +------+---...
Deepak Sharma's user avatar
0 votes
3 answers
2k views

Failed to access postgres data directory on VM

I recently started learning psql, right now I was trying to import some csv spreadsheets into a PostgresQL database using the COPY command. As far as I know, after installing psql, we have a new ...
user9882001's user avatar
3 votes
1 answer
1k views

What are the benefits of using XML over CSV files in regards to data dumps? [closed]

Prologue I've noticed that one benefit of using CSV over XML for data dumps is the fact that fewer disk space is used. See minimal working example below. Let's suppose we have the following dataset. ...
Rodrigo Morales's user avatar
7 votes
1 answer
12k views

Parsing DATE while copying csv file into PostgreSQL table

I have a long series of .csv files, which I want to import into a local database. I believe my query is correct, but there are some problems in parsing DATE and TIMESTAMP columns. PostgreSQL reads ...
baggiponte's user avatar
2 votes
5 answers
4k views

Tsql help - doing a large update from CSV file to a table in ssms

I have been given a csv file with just ids in there and those ids are in a table in ssms with columns that require updating. Out of 90,000 ids in that table, only 83,000 of them require updating from ...
Ujji's user avatar
  • 63
0 votes
0 answers
376 views

Azure Database for MySQL - mysqlimport command

So I'm trying to accomplish the goal of performing a mysqlimport on a Azure MySQL 8 database and I get the following error: mysqlimport: Error: 1227 Access denied; you need (at least one of) the SUPER,...
2189490's user avatar
3 votes
2 answers
2k views

Database Mail has incorrect encoding of CSV attachment

I have a procedure that generates a resultset, which I attach as a CSV file attachment in sp_sendmail. However, the resulting file is encoded UCS-2 LE BOM (according to notepad++). This causes the ...
SqlNightOwl's user avatar
0 votes
1 answer
736 views

importing csv file with ragged right format and " text qualifier

I'm importing csv files in ssis that are in a ragged right format, I've been able to create the columns perfectly fine but for some of the data rows it has included a " text qualifier into the ...
HappyCoder123's user avatar
-1 votes
1 answer
267 views

Can I use a string as an end of line delimiter when importing a .csv to MySQL?

Very basic question but I'm struggling with it. When importing a .csv file to MySQL, is it possible to use any character or string as an end of line delimiter? If so is it possible from the admin ...
Maximilian Travis's user avatar
0 votes
1 answer
593 views

Export mutliple query output into one excel sheet using Redshift and SQL Developer

I am connected to Redshift from SQL Developer 19.2.1. I have a requirement to export multiple query output into a single excel sheet, one query output should be in one Tab. Please advise on how to ...
Vaisakh P's user avatar
0 votes
1 answer
2k views

(PostgreSQL 11) Reading in an array of JSON objects (or composite types) from a CSV file into Postgres

So I am trying to import rows into a table using a CSV file, but have been running into problems I haven't been able to solve. I would like one column of a row to be an array of JSON objects. How I ...
light's user avatar
  • 15
1 vote
2 answers
960 views

BULK INSERT does not fail when file contains commas instead of semicolons when FIRSTROW > 1

I encountered an issue where I received a CSV file that supposed to be delimited by semicolons (;) but was delimited by commas (,). Bulk insert called by sqlcmd did not fail but did not insert either. ...
owl's user avatar
  • 304
1 vote
3 answers
3k views

Japanese characters in VARCHAR column using Japanese_Unicode_CI_AS

I am loading a table with the Japanese names from CSV file that are changing into question marks into SQL Table. The table that is storing these values is a varchar column. I understand that the ...
Nitesh Parasher's user avatar

1
2 3 4 5 6