Tagged Questions

The tag has no wiki summary.

learn more… | top users | synonyms

2
votes
1answer
173 views

Can SQL Server Bulk Copy cause timeouts when querying the table

In SQL Server 2008 R2, can using BulkCopy to upload thousands of rows of data cause problems for queries taking place on the same table? Specifically could it cause the queries to time out? The ...
3
votes
2answers
1k views

Bulk insert with format file: 0 rows affected

When using the BULK INSERT command in SQL Server 2008 it returns: (0 row(s) affected) I am using this command to carry out the bulk insert: BULK INSERT Test FROM 'C:\DataFiles\Tests.dat' ...
0
votes
0answers
413 views

CSV File Into MYSQL Merging - LOAD DATA on Duplicate Key?

Hey Dba Stack Exchange, My question to you is with regards to mysql. Right now I have a medium to small sized csv with 25k lines I need imported into mysql for a client. Problem is, their is only ...
8
votes
2answers
6k views

Optimizing bulk update performance in PostgreSQL

Using PG 9.1 on Ubuntu 12.04. It currently takes up to 24h for us to run a large set of UPDATE statements on a database, which are of the form: UPDATE table SET field1 = constant1, field2 = ...
2
votes
2answers
184 views

SQL Server BULK INSERT

I have a file saved using BCP and I want to know the specification of the table of the database that it needs to be materialized on the database. Is that possible? Thanks
0
votes
1answer
43 views

Reinsert data in new table with new IDs

I have the following two tables: buynsell CREATE TABLE `buynsell` ( `id` VARCHAR(20) NULL DEFAULT NULL, `msg` VARCHAR(255) NULL DEFAULT NULL, `date` VARCHAR(25) NULL DEFAULT NULL, ...
4
votes
2answers
1k views

Index before or after bulk load using load infile?

I have a database with over 1B rows and two columns that are indexed (in addition to the PK). Is it better to have the index pre-defined in the table before the load infile or better to index after ...
2
votes
1answer
3k views

Most efficient way of bulk deleting rows from postgres

I'm wondering what the most efficient way would be to delete large numbers of rows from PostgreSQL, this process would be part of a recurring task every day to bulk import data (a delta of insertions ...
3
votes
1answer
197 views

Which MySQL table to use for read-only, bulk-loaded tables?

So I have a workload on 2 key tables in my application with the following characteristics: READ-ONLY DATA The data is bulk-loaded in a fully consistent manner from a single process on a schedule, ...
5
votes
2answers
1k views

Bulk Data Loading and Transaction Log

im currently working on a project which bulk import data from flat files(csv) about 18 differents files each linking to a specific table through some store procedure. I followed the steps as advised ...
1
vote
1answer
647 views

bulk insert from file

I am often requested to upload data from file (let's say csv) into a Sql Server 2008 db. I would like to use BULK INSERT, but I have no access to the db server file system. Questions is there a ...
9
votes
1answer
2k views

Best way to re-import large amount of data with minimal downtime

I need to import about 500,000 records containing IP lookup (read-only reference) data about once a week (only three int/bigint cols). I don't really want to worry about merging the data with the ...
4
votes
2answers
310 views

fast bulk incrementing in MySQL

I have one big table foobar describing a many-to-many-relation and containing millions of foo's, millions of bar's and every bar having several hundereds of foo's -> billions of rows. CREATE TABLE ...
3
votes
2answers
2k views

Mysql load from infile stuck waiting on hard drive

I've got a windows 7 64 bit machine that I'm using for some load testing of a mysql db. My program uses sqlalchemy to connect and run several load from infile statements on said database. These bulk ...
2
votes
1answer
141 views

After dropping the clustered index, it is still attempting to do a clustered insert [closed]

I am trying to convert data from one database system to another. One of the tables I need to transfer and format contains over 10 million rows. I am running the following script to do it: USE ...
1
vote
1answer
174 views

In a simple table with no complex indexes, does a large number of existing rows slow down an INSERT?

I have this small, simple table: CREATE TABLE `ads_tracker` ( `id` int unsigned NOT NULL AUTO_INCREMENT PRIMARY KEY, `type` smallint unsigned NOT NULL, `reference` smallint unsigned NOT NULL, ...
10
votes
3answers
11k views

Why is 'LOAD DATA INFILE' faster than normal INSERT statements?

I've read an article that mentioned we can achieve 60,000 inserts per second by LOAD DATA IN FILE statement that read from csv files and insert it into database. Why should it differ from normal ...
1
vote
2answers
5k views

What would be the fastest approach to bulk update this mysql table (Innodb)?

I have a table with 100M rows. I am able update my table like 850 updates / minute. How to speed this up? I am using following approach: mysql -uroot < query_file where query_file contains 1M ...
1
vote
1answer
313 views

Importing large file while keeping table available for other operations

We need to import millions of rows in a quite big table. Say like 5 million rows into a table with more than 1 billion rows. After preprocessing the import file we will have update operations and ...
8
votes
1answer
5k views

What specifically does OracleBulkCopy do, and how can I optimize its performance?

To summarize the specifics: We need to stage approximately 5 million rows into a vendor (Oracle) database. Everything goes great for batches of 500k rows using OracleBulkCopy (ODP.NET), but when we ...
3
votes
3answers
269 views

How to send+update on duplicate a big time series to SQL server

I use a SQL Server 2008 I would like to send (and update on duplicate datetime index) a big time series (more than 1000 datapoints and eventually much more than that). What is the best way to do so? I ...
1
vote
1answer
543 views

How do I load them as varbinary(MAX) format into SQL Database?

There are 1000 picture that are jpeg format.How do I load them as varbinary(MAX) format into SQL Database?But at the same time I want the bulk insert functionality.
6
votes
3answers
738 views

Suggestion for Bulk Data Import

We are working on a project which requires import data from Excel spreadsheet on daily basis. The data will be import from pre-define template and what we're thinking, first we will upload data in ...
3
votes
2answers
2k views

How do you copy a table from MySqlServer_A to MySqlServer_B?

I have a daily task that collects data from 3 MySql Servers and then pushes that aggregate data back out to each of the servers deleting the previous days data. All servers are MySql. I would like to ...
2
votes
1answer
238 views

Can I use dbms_redefinition with bulk data transformations for better results?

Here is a situation for bulk loading in Oracle: Schema A { T1, T2, T3, T4} Schema B { TR1, TR2} Where schema B is re-factored from schema A. Both exist in same DB. Schema A has terabytes of ...
5
votes
2answers
2k views

BULK INSERT - best usage

I need to export 150 million rows of only int/bigint columns from one SQL instance to another. I am using BCP queryout to export the rows and BULK INSERT to import the rows to another table. I split ...