0

Overview: In my scenario a user uploads an excel file that needs to be written into database, I am allocating each uploaded file a batch_id that batch_id is written in last column of each row of my table as shown in these pictures File of User 1

enter image description here

With the help of this batch_id column I will be able to segregate that which rows belong to which document.(for example: if batch_id=1 than these 1000 rows belongs to document 1)

Problem: Consider two users user1 and user2 executing this stored procedure at the same time while Stored Procedure is writing batch_id=1 in all the rows which belong to document1,at the same time rows belonging to document2 will gets inserted thus causing a conflict wether to write batch_id=1 or batch_id=2 in the rest of remaining rows.

Imperfect Solution: To Execute Stored Procedure on one document at a time,when its done execute it on document 2 but this will let the user to wait until the current execution finishes which i don't want to happen.

UPDATE:::::::::::::::

The batch_id value is coming from another table MASTER_CDT in this table batch_id is primary key and auto incremented, when a document gets uploaded first a batch_id is generated this batch_id is then inserted into rows corresponding to that batch, the problem is while batch_id(consider batch_id=1) is getting written into current batch rows if another user uploads document then batch_id is changed to value 2 while rows regarding batch 1 are still missing batch_id value but now batch_id=2 will be written to them therefore causing trouble.

5
  • This doesn't sound like a database problem. Your design is lacking information on what is populating the batch field in the spreadsheets, and where that fits into the process of uploading a spreadsheet into the database (and how). Can you add that to the question? Commented Sep 19, 2016 at 14:47
  • 1
    Have you considered using a SEQUENCE object to assign the batch numbers? That will allow parallel inserts for each spreadsheet with different values.
    – Dan Guzman
    Commented Sep 19, 2016 at 14:48
  • @CodyKonior Please have a look at the question i have updated it and instead of considering spreadsheets we can think of db tables only. Commented Sep 20, 2016 at 6:10
  • @DanGuzman Please have a look at the question i have added more info. Commented Sep 20, 2016 at 6:11
  • Does this 'single' stored procedure do the insert into MASTER_CDT and is the batch_id an IDENTITY column? Commented Sep 20, 2016 at 11:49

2 Answers 2

1

You have not posted any code so I'll make some assumptions based on the additional information you added to your question. Once a batch_id is assigned for the upload, the application should use that value when inserting related rows rather than retrieve a batch_id again, which may be different due to concurrency. It seems you may be inserting rows individually and retrieving the batch_id repeatedly, either in code or the proc.

There are a couple of ways to fix the problem. One is to add a @batch_id parameter to the proc and change the code to pass the value from the related MASTER_CDT insert (i.e. returned with SCOPE_IDEINITY()) for each spreadsheet row insert.

A more elegant solution is to pass the spreadsheet contents as a table-valued parameter like the example below. Not only will this address the concurrency problem, it will be much more efficient than individual inserts. Whether or not this is feasible depends on the language/API you are using.

--create table type for TVP
CREATE TYPE dbo.EmployeeSpreadsheetRows AS TABLE(
      employee_id int
    , name nvarchar(100)
    , email varchar(255)
    , phone_number varchar(20)
    , date_of_birth date
    , address nvarchar(100)
    , department nvarchar(100)
    , manager nvarchar(100)
    , validation_message nvarchar(1000)
    );
GO


CREATE PROC dbo.InsertSpreadsheet
      @FileName varchar(255) --values needed for MASTER_CDT table insert
    , @EmployeeSpreadsheetRows dbo.EmployeeSpreadsheetRows READONLY --spreadsheet rows
AS
SET XACT_ABORT ON;

BEGIN TRY
    BEGIN TRAN;
    INSERT INTO dbo.MASTER_CDT(FileName) VALUES(@FileName);
    DECLARE @batch_id int = SCOPE_IDENTITY();
    INSERT INTO EMPLOYEE_SPREADSHEET(
          employee_id
        , name
        , email
        , phone_number
        , date_of_birth
        , address
        , department
        , manager
        , validation_message
        , batch_id
        )
    SELECT
          employee_id
        , name
        , email
        , phone_number
        , date_of_birth
        , address
        , department
        , manager
        , validation_message
        , @batch_id
    FROM @EmployeeSpreadsheetRows;
    COMMIT;
END TRY
BEGIN CATCH
    IF @@TRANCOUNT > 1 ROLLBACK;
    THROW;
END CATCH;
GO
0

If I'm hearing what you're saying correctly, consider making your batch_id "match" an uploaded document by adding in your procedure something like:

declare @batch_id uniqueidentifier 
select @batch_id = newid()
insert into main_table (employee_id, name, email, ... batch_id) values (101,'Max Chan','[email protected]', @batch_id)
insert into main_table (employee_id, name, email, ... batch_id) values (102,'Next User','[email protected]', @batch_id)

That way, the batch_id will match the uploaded document. And, no batch_ids will be the same using the GUID approach.

2
  • 1
    2 different connections could get the same batch_id with this method if the both happen to execute the SELECT before the insert.
    – Dan Guzman
    Commented Sep 19, 2016 at 14:41
  • Each batch might have 40 thousand rows and for those 40 thousand rows i want same batch id that's way i will segregate batches. Commented Sep 20, 2016 at 6:13

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.