How to deploy SSIS package with XML config, using SQL agent

Question

I wonder if anyone can help me with a question.

Context I've created a database which needs a front end in MS Excel. When the user presses upload button I want it to start an SSIS package which moves the data into some tmp tables then runs a couple of stored procs to move the data into the correct tables. I'm struggling to deploy my package successfully.

Hurdles I've already jumped over

1. The SSIS package is functional in debug mode.
2. I've got a SQL Agent job which can run the package on the integration services server file system.
3. I've managed to create a way of the users starting the job from excel and inheriting SQLAgentOperatorRole through the very convoluted process of creating certificates and a new stored proc, I called it spTriggerJob [with parameter for job name], which then runs EXEC msdb.dbo.sp_start_job @job_name = @strJob with the correct privileges.
4. I've created and implemented a proxy with permissions to access the network file server.
5. I've created XML configurations to pass the required variables (e.g. the filepath of the file to be uploaded) and a way of updating them on-the-fly.

Problem 1 When I run the package manually from SSMS I can load the XML configurations successfully. However I haven't been able to do this from the SQL Agent job. If I go into job properties > steps > edit > configurations tab > add the only options I get are C:\ D:\ and a couple of obscure/unhelpful network drives. Ideally I want to point it at a full UNC filepath. Is it necessary to map the drive on the server?

I suspect there is a more optimal way of deploying my SSIS package which circumvents this. Can anyone suggest a better way? I'm fairly new to all this so don't make any assumptions as to my existing knowledge!

Problem 2 When I use my spTriggerJob [name of job], I see a weird message in the job history. It says "The job succeeded. The Job was invoked by user " then it gives the Windows account login of one of the network administrators. However I've been able to verify that the permissions of the proxy (which have nothing to do with Mr Network Admin) are applied correctly by using it to create an entry in a test table and recording the user name. However if I call msdb.dbo.sp_start_job directly, then the login which appears in the job history message is my own. This seems wrong! Does anyone know how to resolve this?

Edit: I figured out problem 2. It is because 'spTriggerJob' has to have 'WITH EXECUTE AS OWNER'. Whilst I thought I was owner, in fact it is Mr Network Admin. The confusing bit was that the job was running with the correct privileges inherited from the proxy, which threw me off a bit.

Required solution I need to reach a point where the package is run on an ad hoc basis at the click of a button, with different configuration settings each time it is run. Surely it can't be that hard!!!

Thanks in advance for your help.

Cheers, Andy

Answer 1

If you want to store the configuration file and/or the file you need to "pick up" via the SSIS package on a UNC path, SQL Server is going to need permissions to that direct UNC path. This permission is based on your service account being used for SQL Server Agent. If you are in a domain environment, then it is best to have your service run as a domain account and it will allow you to easily provide access to network resources.

If this is a configuration that is meant to change often and I would not be the one to do it necessarily every time, then I might chose to store the configuration in SQL Server. There are a few examples out there online that show how to do this, just one example here.