Getting a "Request to run job refused because the job is already running" When no job is running?

Question

So have an ETL job that runs through integration services that I start with a SQL job name StartStageStore. The last step of this job is to start another job and report success. The job this calls to is referred to as LoopStageStore. All LoopStageStore does is execute a stored procedure that Restarts the StartStageStore job all over again this way we are constantly pulling source data into our system. This job has been running for sever weeks now without failure. Until the last two nights I started getting this error "Executed as user: AgentUser. SQLServerAgent Error: Request to run job Stage/Store AgentUser refused because the job is already running from a request by User AgentUser. [SQLSTATE 42000] (Error 22022). The step failed."

This should not be possible as the run StartStageStore job finishes and is no longer running when LoopStageStore has started.

Here we can see the last step executed at 12:19:14.

And the Next Job Fails at 12:19:15 Saying that StartStageStore is currently running. I already have a workaround for this where I just throw a Delay on the Second job. I'm just wondering if anyone knows why this would happen.

BEGIN
WAITFOR DELAY '00:01';
    EXECUTE dbo.LoopStageStore
END; GO

Answer 1

Alright so the problem I was having was that the write time to the log file for job completion occasionally takes longer than it takes to run the stored procedure that restarts this loop.

The solution to this is my original workaround.

In case anyone wants to use it here is the code.

BEGIN
WAITFOR DELAY '00:01';
    EXECUTE dbo.LoopStageStore
END; GO

Just make it wait a little bit.