Questions tagged [csv]

Questions on how to import into, extract data from, or exchange data between databases in CSV format (plain text using commas to separate values).

Filter by
Sorted by
Tagged with
0
votes
0answers
20 views

PostgreSQL 10 - create partitions on column that needs to be calculated after importing a csv

I am in the situation where I need to create a table which also has a time column. This table is filled with data taken from a CSV, but the csv has no time column. Only after importing the data inside ...
0
votes
1answer
50 views

ORA-06502: 'character string buffer too small' when writing to a csv

I have some tables that i want to export. I'm using this package (I called it "CSV"): https://www.williamrobertson.net/documents/refcursor-to-csv.shtml I was able to write to CSV files ...
0
votes
1answer
24 views

attach csv file on email sent from postgres

I have the following code to send emails and wanted to see if it's possible to attach a csv file? CREATE OR REPLACE FUNCTION public.send_email_html(text, text, text, text) RETURNS integer LANGUAGE ...
2
votes
2answers
34 views

using SQLCMD to export/extract data withot trailing spaces

I am trying to extract data from some of my tables using sqlcmd utility as SSMS does not generate appropriate csv-s for me. My issue: all csv extract includes trailing spaces according to table ...
2
votes
1answer
62 views

Copy csv file into different schema [closed]

I need to import data from a csv file which from database a schema a into another database b schema b. I used the pgAdmin running the export and when I import the data using the same tool pgAdmin and/...
1
vote
0answers
29 views

Export and split QUERY results in multiple files

I would like to know if it's possible to export and split query results in multiple files. Let's assume I have a query returning 100 results and I want to save the results in 5 different csv files (20 ...
0
votes
0answers
121 views

Persistent problems importing CSV files via pgAdmin 4 to PostgreSQL 12 on Mac OS - Failing with Exit Code -6

I've been trying for several days to import CSV files into a PostgreSQL 12 database hosted on Mac OS 10.15.5, using the Import/Export tool in pgAdmin 4. I am consistently getting an error message ...
0
votes
0answers
29 views

Attempted to import .csv into mySQL using workbench, but only imports 7 rows?

To start, I'm still a relative SQL noob so this question could have an obvious answer that I'm just missing. I'm trying to start a big practice project in my spare time using a dataset which has ~...
0
votes
1answer
41 views

MS SQL Server: Import Flat File with auto-increment ID

So I've downloaded a sample database. It originally didn't have an ID column so I inserted one in through Excel. Anyways, I went through DatabaseName > Tasks > Import Flat File and the usual ...
0
votes
2answers
62 views

RDBMS: Proper way to store data - comma separated variables or in different fields or tables?

Fundamental question that I cannot seem to find an answer on. I have a database that stores line items from receipts along with the username and receipt number. Right now the lineItem column is just ...
0
votes
1answer
279 views

PostgreSQL: date/time field value out of range

Software: Linux cjz-eshop1-p 5.4.0-33-generic #37-Ubuntu SMP Thu May 21 12:53:59 UTC 2020 x86_64 x86_64 x86_64 GNU/Linux psql (PostgreSQL) 12.3 (Ubuntu 12.3-1.pgdg18.04+1) I am getting errors due to ...
0
votes
2answers
141 views

How to import multiple CSV files to a postgres table using pgadmin or other method?

I'm looking to import a folder of over 100+ csv files into one table on PostgreSQL using pgadmin. I have the following script but is there a way to change the file path to all csvs? COPY schema.table ...
1
vote
1answer
33 views

SQL (Redshift) SELECT COUNT from CSV column

Hi exchange community, Problem statement and details: I am trying to query a column that contains CSV data to produce a count of instances of each value in the table. There are a total of 14 ...
0
votes
1answer
28 views

How can I monitor the import progress of a large CSV File in mysql?

I am trying to import an existing CSV file into my database and I want to see the progress. I usually use pipe viewer when I do my sql files but for csv files I cannot figure out how to do it. I use ...
0
votes
1answer
65 views

MySQL Workbench: I need to update values in a table based on values from a csv

Newbie Problem, but I've tried creating a new table and importing the csv to that then updating, but I have been having trouble with that, any guidance would be greatly appreciated. I have a csv file ...
0
votes
0answers
24 views

CommandLine for Exporting All MS SQL Tables to CSV

On my Mac, I have a docker container running an Ubuntu version of MS SQL. I have a DB running in it and I can access it via Azure Data Studio, and I can view each table individually and export it to ...
0
votes
1answer
50 views

How to Export MS SQL DB from Docker-Linux to Access format

I have a MS SQL database which is currently up and running on my Mac within a Docker container that is hosting the Ubuntu Linux version of MS SQL Server. I need to export the database to a format ...
0
votes
2answers
435 views

Why BULK INSERT insert data in random order?

I have a csv file with 350,000 rows. Would like to insert data to temporary table in same order as is in csv file. I am trying to BULK INSERT using: BULK INSERT ##AllRows FROM @FilePath WITH ...
0
votes
1answer
52 views

Mysql CSV import when using LOAD DATA LOCAL INFILE only imports half of the rows

I am doing an import with below code and it works but only imports every second row (1,3,5,7...) and only imports half the total csv contents at the end. LOAD DATA LOCAL INFILE "'.$file.'" INTO TABLE ...
0
votes
0answers
25 views

Mongo compass import csv with accentuation

I'm trying to solve the following problem ... When trying to import data from a csv file into my collection, it happens that in the database some words with accents are left with question marks. ...
1
vote
0answers
20 views

How to migrate data from a large amount of csv files with time x space measurements to a database that needs to be available for analysis [closed]

I have made a simulation experiment to obtain time series measurements at different points along a power line. These measurements were done for different types of power lines in different test ...
0
votes
0answers
211 views

Import from csv file to SQL table - Double quotes on the very end

I was able to import a .CSV file into my sql table, here is my BULK INSERT script: BULK INSERT TLS_Sysprocess FROM 'D:\Dennis\DevAPPServers2.csv' WITH ( FirstRow=2 ,FIELDTERMINATOR = '","' ,...
0
votes
1answer
64 views

Postgres randomly fails silently on huge CSV import

With Postgres 10.12 on Windows 10, I am importing at 67 GB CSV file into a database using a simple COPY query: COPY table_name FROM 'c:/temp/data.csv' WITH (FORMAT csv, HEADER); The import fails ...
0
votes
1answer
41 views

CSV or DB is quicker to read several lines from a file with 1,5 billion lines

I have a program which would read several lines according to the indices of lines within a CSV file with 1,5 billion lines(using c fgets, or is there any tool in C faster than fgets?). Compare with ...
0
votes
1answer
25 views

How often is Postgres parsing a foreign table based on file_fdw?

I have a foreign table defined with file_fdw, which reads the data from a CSV file. Postgresql parses the CSV file and converts the date for example from YYYYMMDD to YYYY-MM-DD. This works fine. My ...
0
votes
0answers
8 views

MySQL 8.0 appending unknown character to column

I'm inserting a CSV files into MySQL 8.0 using Python 3.7: bulk_insert_sql = "LOAD DATA LOCAL INFILE filename.csv INTO TABLE table_name FIELDS TERMINATED BY '|' LINES TERMINATED BY '\n' IGNORE 1 ...
0
votes
1answer
418 views

Mysql output to csv putting column names on the bottom

I'm trying to export the results of the query listed below to csv and have the column names in the first row. However, when I run the code and open up the csv the header is listed as the last row. I ...
0
votes
0answers
34 views

LOAD DATA INFILE NULL are enclosed not loaded

I have a CSV file that I exported from Oracle database. All the integer columns are not enclosed, but all the string columns are enclosed with " I'm using MySQL 8. So I used the following command ...
0
votes
2answers
33 views

Ram & Concurrency: MySQL InnoDB LOAD INLINE FILE on 50 30GB CSV files

As the title suggests I'm moving around a lot of data. This is in an environment with a 24 core CPU and 2TB of M.2 storage. My first question is: given I have 16GB of ram, does that mean it's ...
1
vote
1answer
653 views

Copy large PostgreSQL table to multiple csv files with limited row number

Is there a way, using a sql function or an already existing psql function, to copy a very large table into multiple csv files and specify the maximum number of rows allowed in the output files ? For ...
0
votes
2answers
69 views

How to automate the detection of the column type of a csv file for a table creation script?

I am new to pgAdmin and Azure database. I have a huge csv file with around 220 columns and I want to create a table out of it in pgAdmin4 to push it to Azure. However I don't know how to detect ...
1
vote
2answers
59 views

How to create a csv with SQLCMD which can be imported into another database system

I may be doing this all the wrong way so please point me in the right direction if I am missing something. I have an Ubuntu server with SQLCMD installed and a Windows Server with a MSSQL database. I ...
0
votes
1answer
222 views

How to add multiple query results into one csv file?

By the help of the members of this community I have created a script which shows recent T-log backup for each database in the server. Instead of sp_MSforeachdb I used a customized script written by ...
0
votes
0answers
37 views

Import folder with Multiple CSV's into SSMS w/ out SSIS

I am wondering what's the best and most effective way to import csv's stored in a local folder into DB w/ in SSMS. Folder path- C:\Users\ngutierrez\Desktop\Open_Air_Tables Folder contains about 30 ...
1
vote
1answer
1k views

exporting a sql view to a csv file

Whilst I can just about script to pull data via queries, I'm finding it very difficult to figure out how to export the results of a query to a .csv file and save in a network storage location (novice ...
0
votes
0answers
97 views

Errors with trying to run a stored procedure with local file output

I'm trying to run a stored procedure that I'm passing some variables to. The output is being writted to a CSV file on a local file share. The CSV file gets created, but I'm hitting some errors ...
-1
votes
3answers
828 views

Sub-select from comma separated list in WHERE clause

I have tables like so: TableA id id_link_list 1 1,2 TableB id info 1 blah 2 blah-more I'm struggling to include the TableA.id_link_list column (which is already a comma ...
0
votes
1answer
1k views

Importing CSV file into a PostgreSQL table

I am using Python with pandas to import a CSV file into a table in Postgres import pandas as pd import psycopg2 from sqlalchemy import create_engine df = pd.read_csv('products.csv', sep=';', ...
0
votes
3answers
1k views

Extracting data from extremely large csv files [closed]

I have a 40gb csv file with over 60 million rows for data analysis. Each row has a unique identifier (some numbers). For example, the first row's unique identifier will repeat approximately 150,000 ...
0
votes
1answer
29 views

Best practices for storing multiple SQL tables per user (built from uploaded csv file?)

I have a few users who will be uploading a few types of predefined CSV files. I need to store these as sql tables to be able to run queries on this data for a particular user, however that means there'...
1
vote
0answers
19 views

Custom Export from power BI service

I'm trying to make a custom button that exports filtered tables to a csv and downloads them to my computer. I am aware that you can do this with the '...' button but this functionality is for other ...
0
votes
0answers
68 views

Can I force a LOAD DATA LOCAL statement to abort if an error is not loaded?

I am trying to load a CSV file into a MySQL table with the load data local statement. After loading it, the statement reports Query OK, 1030 rows affected, 7314 warnings (0.03 sec) Records: 1043 ...
0
votes
0answers
168 views

Writing CSV directly to a BYTEA column from PostgreSQL tables

Assuming these two tables: CREATE TABLE data (day DATE, kpi INTEGER); CREATE TABLE reports (csv BYTEA); Is it possible to (pseudocode): INSERT INTO reports (csv) (SELECT to_csv(SELECT day, kpi FROM ...
1
vote
1answer
255 views

InfluxDB batch insert only inserts the last line

I'm trying to insert some lines into a InfluxDB instance (running on Windows 7 x64) from both a .txt and a .csv file, but it's only inserting the last line on the file. I'm currently using the ...
0
votes
1answer
21 views

Load flat file into a predetermined table

Scenario Table created Data in flat file loaded to the table but with errors. please advise how to fix it (a) 'address' is split to 3 columns causing mismatch between the source and destination (b)...
-1
votes
1answer
451 views

How to normalize CSV invoice data?

For flat data which comes in with fields like: name address status class version user mode CSV files are regularly received, with the above as typical columns. Some fields, such as "class"...
0
votes
2answers
2k views

MongoDB CSV Export: One Array/Column & Array Index on rows, or vice versa

I have a MongoDB schema that looks like this: const batchSchema = mongoose.Schema({ _id: mongoose.Schema.Types.ObjectId, time: {type: [Number]}, tmp: {type: [Number]}, hum: {type: [Number]}, ...
0
votes
1answer
39 views

csv engine and mysql start

my question is about speed when perform service mysql start. with many tables (some not often used). what engines consumes more or less start time ? example: innodb --> the more tables, the more ...
1
vote
1answer
222 views

\COPY not returning count psql

I am trying to copy CSV data into tables. Also I want to print the count. The command I'm using is: echo "begin; delete from a; \copy a from a.csv CSV HEADER; end;" | psql -hlocalhost -dpostgres -...
2
votes
2answers
2k views

Can't import flat file into SqlServer

I've got this tab-delimited flat file with fifty columns and two million rows. I am trying to import it into Sql Server 2017 Express. (I first tried working with it in Excel... hopeless.) I see two ...

1
2 3 4 5