The bulk tag has no wiki summary.
2
votes
2answers
107 views
SQL Server BULK INSERT
I have a file saved using BCP and I want to know the specification of the table of the database that it needs to be materialized on the database. Is that possible?
Thanks
0
votes
1answer
28 views
Reinsert data in new table with new IDs
I have the following two tables:
buynsell
CREATE TABLE `buynsell` (
`id` VARCHAR(20) NULL DEFAULT NULL,
`msg` VARCHAR(255) NULL DEFAULT NULL,
`date` VARCHAR(25) NULL DEFAULT NULL,
...
4
votes
3answers
125 views
Index before or after bulk load using load infile?
I have a database with over 1B rows and two columns that are indexed (in addition to the PK).
Is it better to have the index pre-defined in the table before the load infile or better to index after ...
2
votes
1answer
183 views
Most efficient way of bulk deleting rows from postgres
I'm wondering what the most efficient way would be to delete large numbers of rows from PostgreSQL, this process would be part of a recurring task every day to bulk import data (a delta of insertions ...
3
votes
1answer
80 views
Which MySQL table to use for read-only, bulk-loaded tables?
So I have a workload on 2 key tables in my application with the following characteristics:
READ-ONLY DATA
The data is bulk-loaded in a fully consistent manner from a single process on a schedule, ...
4
votes
2answers
223 views
Bulk Data Loading and Transaction Log
im currently working on a project which bulk import data from flat files(csv) about 18 differents files each linking to a specific table through some store procedure.
I followed the steps as advised ...
1
vote
1answer
274 views
bulk insert from file
I am often requested to upload data from file (let's say csv) into a Sql Server 2008 db.
I would like to use BULK INSERT, but I have no access to the db server file system.
Questions
is there a ...
8
votes
1answer
1k views
Best way to re-import large amount of data with minimal downtime
I need to import about 500,000 records containing IP lookup (read-only reference) data about once a week (only three int/bigint cols).
I don't really want to worry about merging the data with the ...
4
votes
2answers
175 views
fast bulk incrementing in MySQL
I have one big table foobar describing a many-to-many-relation and containing millions of foo's, millions of bar's and every bar having several hundereds of foo's -> billions of rows.
CREATE TABLE ...
3
votes
2answers
867 views
Mysql load from infile stuck waiting on hard drive
I've got a windows 7 64 bit machine that I'm using for some load testing of a mysql db. My program uses sqlalchemy to connect and run several load from infile statements on said database. These bulk ...
2
votes
1answer
101 views
After dropping the clustered index, it is still attempting to do a clustered insert
I am trying to convert data from one database system to another. One of the tables I need to transfer and format contains over 10 million rows.
I am running the following script to do it:
USE ...
1
vote
1answer
103 views
In a simple table with no complex indexes, does a large number of existing rows slow down an INSERT?
I have this small, simple table:
CREATE TABLE `ads_tracker` (
`id` int unsigned NOT NULL AUTO_INCREMENT PRIMARY KEY,
`type` smallint unsigned NOT NULL,
`reference` smallint unsigned NOT NULL,
...
7
votes
3answers
5k views
Why is 'LOAD DATA INFILE' faster than normal INSERT statements?
I've read an article that mentioned we can achieve 60,000 inserts per second by LOAD DATA IN FILE statement that read from csv files and insert it into database.
Why should it differ from normal ...
1
vote
2answers
2k views
What would be the fastest approach to bulk update this mysql table (Innodb)?
I have a table with 100M rows. I am able update my table like 850 updates / minute. How to speed this up?
I am using following approach:
mysql -uroot < query_file
where query_file contains 1M ...
1
vote
1answer
236 views
Importing large file while keeping table available for other operations
We need to import millions of rows in a quite big table.
Say like 5 million rows into a table with more than 1 billion rows.
After preprocessing the import file we will have update operations and ...
7
votes
1answer
2k views
What specifically does OracleBulkCopy do, and how can I optimize its performance?
To summarize the specifics: We need to stage approximately 5 million rows into a vendor (Oracle) database. Everything goes great for batches of 500k rows using OracleBulkCopy (ODP.NET), but when we ...
3
votes
3answers
201 views
How to send+update on duplicate a big time series to SQL server
I use a SQL Server 2008
I would like to send (and update on duplicate datetime index) a big time series (more than 1000 datapoints and eventually much more than that).
What is the best way to do so?
I ...
1
vote
1answer
395 views
How do I load them as varbinary(MAX) format into SQL Database?
There are 1000 picture that are jpeg format.How do I load them as varbinary(MAX) format into SQL Database?But at the same time I want the bulk insert functionality.
6
votes
3answers
635 views
Suggestion for Bulk Data Import
We are working on a project which requires import data from Excel spreadsheet on daily basis. The data will be import from pre-define template and what we're thinking, first we will upload data in ...
3
votes
2answers
1k views
How do you copy a table from MySqlServer_A to MySqlServer_B?
I have a daily task that collects data from 3 MySql Servers and then pushes that aggregate data back out to each of the servers deleting the previous days data. All servers are MySql. I would like to ...
2
votes
1answer
208 views
Can I use dbms_redefinition with bulk data transformations for better results?
Here is a situation for bulk loading in Oracle:
Schema A { T1, T2, T3, T4}
Schema B { TR1, TR2}
Where schema B is re-factored from schema A. Both exist in same DB. Schema A has terabytes of ...
5
votes
2answers
2k views
BULK INSERT - best usage
I need to export 150 million rows of only int/bigint columns from one SQL instance to another. I am using BCP queryout to export the rows and BULK INSERT to import the rows to another table.
I split ...
