Join tables where the table name changes every day

Question

I need some help in figuring out how to join tables to my query. The table names change every day as the date is used in the table name and older tables are dropped as they are no longer needed.

My query looks for "scans" that have data in the database and are in the process of being processed (for a lack of a better word).

My current query is:

USE RetinaCSDatabase
SELECT S.ScanID [Scan ID], S.PolicyName [Scan], DATEADD(hh, +2, S.StartTime) [Scan Start], A.AssetName [Scanner], COUNT(AIPS.AssetIPID) [Hosts], S.TransactionGroup [Transaction Group ID]
FROM Scan S (NOLOCK)
LEFT JOIN Scanner SC (NOLOCK) ON S.ScannerID = SC.ScannerID
LEFT JOIN Asset A (NOLOCK) ON SC.AssetID = A.AssetID
LEFT JOIN Asset_IP_Scan AIPS (NOLOCK) ON S.ScanID = AIPS.ScanID
WHERE S.ProcessStatus < 2
GROUP BY S.ScanID, S.PolicyName, S.StartTime, A.AssetName, S.TransactionGroup
ORDER BY S.StartTime ASC, S.PolicyName ASC, A.AssetName ASC

What I need to join is the table(s) with the data still requiring processing. Those tables start with the name "evt_scan_" and then the date. So, a full table name would be "evt_scan_20171222" for example. The data for a scan performed on the 22nd of December 2017 will have data in the table "evt_scan_20171222" and possibly in the following days table, "evt_scan_20171223". So, I need my query first to check if the table exists and then join it to do a count and also to check if the following days table exists, join it to do another count and add up the 2 counts to show the outstanding records requiring processing.

If the table names were static I would join them with this query:

LEFT JOIN evt_scan_20171222 evt1 ON S.TransactionGroup = evt1.evtTransactionGroup

It would also require a WHERE statement returning only records where evtStatus = 0.

Thanks

Answer 1

Use a synonym.

USE YourDatabaseName;
DECLARE @sql nvarchar(max) = N'CREATE SYNONYM
  dbo.EventScan FOR dbo.evt_scan_'
  + CONVERT(CHAR(8), GETDATE(), 112);
EXEC sys.sp_executesql @sql;

Then your code can always reference dbo.EventScan.

Now, schedule a job to run every night at 12:00:01 AM (or create a DDL trigger on table creation and when the new table name matches the expected pattern, or put code in whatever creates those new tables each day) that does this:

USE YourDatabaseName;
DROP SYNONYM dbo.EventScan;
— repeat code from above

Answer 2

please try this query built with sp_executesql

declare @tableName nvarchar(100) = 'evt_scan_20171222';
declare @SQLString nvarchar(1000)
SET @SQLString =  N'
    SELECT S.ScanID [Scan ID], S.PolicyName [Scan], 
        DATEADD(hh, +2, S.StartTime) [Scan Start], A.AssetName [Scanner], 
        COUNT(AIPS.AssetIPID) [Hosts], S.TransactionGroup [Transaction Group ID]
    FROM Scan S (NOLOCK)
        LEFT JOIN Scanner SC (NOLOCK) ON S.ScannerID = SC.ScannerID
        LEFT JOIN Asset A (NOLOCK) ON SC.AssetID = A.AssetID
        LEFT JOIN Asset_IP_Scan AIPS (NOLOCK) ON S.ScanID = AIPS.ScanID
        LEFT JOIN ' + @tableName + ' evt1 ON S.TransactionGroup = evt1.evtTransactionGroup
    WHERE S.ProcessStatus < 2
    GROUP BY S.ScanID, S.PolicyName, S.StartTime, A.AssetName, S.TransactionGroup
    ORDER BY S.StartTime ASC, S.PolicyName ASC, A.AssetName ASC'
execute sp_executesql @sqlstring

But actually dynamically tables names is not best practice.

Update

Thanks to Aaron Bertrand in comments. Changes:

declare @SQLString nvarchar(100) //- not enough for hande sql string.

Answer 3

Another approach (along similar lines of Aaron Bertrand's approach), but dynamically generating a VIEW instead of a SYNONYM. Your question indicated that your data could be on multiple files across different days. You also indicated that older files are deleted when no longer needed. The following code comes courtesy of Steve Oaks. The EXEC is commented out.

DECLARE @TableName NVARCHAR(400)
DECLARE @DynSQL NVARCHAR(MAX)

SET @DynSQL = ''

DECLARE cursor1 CURSOR
FOR
SELECT NAME
FROM sys.tables
WHERE NAME LIKE 'evt_scan_%'

OPEN cursor1

FETCH NEXT
FROM cursor1
INTO @TableName

WHILE @@FETCH_STATUS = 0
BEGIN
    -- Add the select code.
    SET @DynSQL = @DynSQL + 'Select * from ' + @TableName

    FETCH NEXT
    FROM cursor1
    INTO @TableName

    -- If the loop continues, add the UNION ALL statement.
    IF @@FETCH_STATUS = 0
    BEGIN
        SET @DynSQL = @DynSQL + ' UNION ALL '
    END
END

CLOSE cursor1

DEALLOCATE cursor1

SET @DynSQL = 'CREATE VIEW EVT_SCAN AS ' + @DynSQL

PRINT @DynSQL

--EXEC sp_executesql @DynSQL