Group daily schedule into [Start date; End date] intervals with the list of week days

Question

I need to convert data between two systems.

First system stores schedules as a plain list of dates. Each date that is included in the schedule is one row. There can be various gaps in the sequence of dates (weekends, public holidays and longer pauses, some days of the week may be excluded from the schedule). There can be no gaps at all, even weekends can be included. The schedule can be up to 2 years long. Usually it is few weeks long.

Here is a simple example of a schedule that spans two weeks excluding weekends (there are more complicated examples in the script below):

+----+------------+------------+---------+--------+
| ID | ContractID |     dt     | dowChar | dowInt |
+----+------------+------------+---------+--------+
| 10 |          1 | 2016-05-02 | Mon     |      2 |
| 11 |          1 | 2016-05-03 | Tue     |      3 |
| 12 |          1 | 2016-05-04 | Wed     |      4 |
| 13 |          1 | 2016-05-05 | Thu     |      5 |
| 14 |          1 | 2016-05-06 | Fri     |      6 |
| 15 |          1 | 2016-05-09 | Mon     |      2 |
| 16 |          1 | 2016-05-10 | Tue     |      3 |
| 17 |          1 | 2016-05-11 | Wed     |      4 |
| 18 |          1 | 2016-05-12 | Thu     |      5 |
| 19 |          1 | 2016-05-13 | Fri     |      6 |
+----+------------+------------+---------+--------+

ID is unique, but it is not necessarily sequential (it is primary key). Dates are unique within each Contract (there is unique index on (ContractID, dt)).

Second system stores schedules as intervals with the list of week days that are part of the schedule. Each interval is defined by its start and end dates (inclusive) and a list of week days that are included in the schedule. In this format you can efficiently define repetitive weekly patterns, such as Mon-Wed, but it becomes a pain when a pattern is disrupted, for example by public holiday.

Here is how the simple example above will look like:

+------------+------------+------------+----------+----------------------+
| ContractID |  StartDT   |   EndDT    | DayCount |       WeekDays       |
+------------+------------+------------+----------+----------------------+
|          1 | 2016-05-02 | 2016-05-13 |       10 | Mon,Tue,Wed,Thu,Fri, |
+------------+------------+------------+----------+----------------------+

[StartDT;EndDT] intervals that belong to the same Contract should not overlap.

I need to convert data from the first system into the format used by the second system. At the moment I'm solving this on the client side in C# for the single given Contract, but I'd like to do it in T-SQL on the server side for bulk processing and export/import between servers. Most likely, it could be done using CLR UDF, but at this stage I can't use SQLCLR.

The challenge here is to make the list of intervals as short and human-friendly as possible.

For example, this schedule:

+-----+------------+------------+---------+--------+
| ID  | ContractID |     dt     | dowChar | dowInt |
+-----+------------+------------+---------+--------+
| 223 |          2 | 2016-05-05 | Thu     |      5 |
| 224 |          2 | 2016-05-06 | Fri     |      6 |
| 225 |          2 | 2016-05-09 | Mon     |      2 |
| 226 |          2 | 2016-05-10 | Tue     |      3 |
| 227 |          2 | 2016-05-11 | Wed     |      4 |
| 228 |          2 | 2016-05-12 | Thu     |      5 |
| 229 |          2 | 2016-05-13 | Fri     |      6 |
| 230 |          2 | 2016-05-16 | Mon     |      2 |
| 231 |          2 | 2016-05-17 | Tue     |      3 |
+-----+------------+------------+---------+--------+

should become this:

+------------+------------+------------+----------+----------------------+
| ContractID |  StartDT   |   EndDT    | DayCount |       WeekDays       |
+------------+------------+------------+----------+----------------------+
|          2 | 2016-05-05 | 2016-05-17 |        9 | Mon,Tue,Wed,Thu,Fri, |
+------------+------------+------------+----------+----------------------+

,not this:

+------------+------------+------------+----------+----------------------+
| ContractID |  StartDT   |   EndDT    | DayCount |       WeekDays       |
+------------+------------+------------+----------+----------------------+
|          2 | 2016-05-05 | 2016-05-06 |        2 | Thu,Fri,             |
|          2 | 2016-05-09 | 2016-05-13 |        5 | Mon,Tue,Wed,Thu,Fri, |
|          2 | 2016-05-16 | 2016-05-17 |        2 | Mon,Tue,             |
+------------+------------+------------+----------+----------------------+

I tried to apply a gaps-and-islands approach to this problem. I tried to do it in two passes. In the first pass I find islands of simple consecutive days, i.e. the end of the island is any gap in the sequence of days, be it weekend, public holiday or something else. For each such found island I build a comma-separated list of distinct WeekDays. In the second pass I group found islands further by looking at the gap in the sequence of week numbers or a change in the WeekDays.

With this approach each partial week ends up as an extra interval as shown above, because even though week numbers are consecutive, the WeekDays change. Besides, there can be regular gaps within a week (see ContractID=3 in sample data, which has data only for Mon,Wed,Fri,) and this approach would generate separate intervals for each day in such schedule. On the bright side, it generates one interval if the schedule doesn't have any gaps at all (see ContractID=7 in the sample data that includes weekends) and in that case it doesn't matter if the start or end week is partial.

Please see other examples in the script below to get a better idea of what I'm after. You can see that quite often weekends are excluded, but any other days of the week could also be excluded. In example 3 only Mon, Wed and Fri are part of the schedule. Besides, weekends can be included, as in example 7. The solution should treat all days of the week equally. Any day of the week can be included or excluded from the schedule.

To verify that the generated list of intervals describes the given schedule correctly you can use the following pseudo-code:

• loop through all intervals
• for each interval loop through all calendar dates between Start and End dates (inclusive).
• for each date check if its day of the week is listed in the WeekDays. If yes, then this date is included in the schedule.

Hopefully, this clarifies in what cases a new interval should be created. In examples 4 and 5 one Monday (2016-05-09) is removed from the middle of the schedule and such schedule can't be represented by a single interval. In example 6 there is a long gap in the schedule, so two intervals are needed.

Intervals represent weekly patterns in the schedule and when a pattern is disrupted/changed the new interval has to be added. In example 11 first three weeks have a pattern Tue, then this pattern changes to Thu. As a result we need two intervals to describe such schedule.

---

I'm using SQL Server 2008 at the moment, so solution should work in this version. If a solution for SQL Server 2008 can be simplified/improved using features from later versions, that's a bonus, please show it as well.

I have a Calendar table (list of dates) and Numbers table (list of integer numbers starting from 1), so it is OK to use them, if needed. It is also OK to create temporary tables and have several queries that process data in several stages. The number of stages in an algorithm has to be fixed though, cursors and explicit WHILE loops are not OK.

---

Script for sample data and expected results

-- @Src is sample data
-- @Dst is expected result

DECLARE @Src TABLE (ID int PRIMARY KEY, ContractID int, dt date, dowChar char(3), dowInt int);
INSERT INTO @Src (ID, ContractID, dt, dowChar, dowInt) VALUES

-- simple two weeks (without weekend)
(110, 1, '2016-05-02', 'Mon', 2),
(111, 1, '2016-05-03', 'Tue', 3),
(112, 1, '2016-05-04', 'Wed', 4),
(113, 1, '2016-05-05', 'Thu', 5),
(114, 1, '2016-05-06', 'Fri', 6),
(115, 1, '2016-05-09', 'Mon', 2),
(116, 1, '2016-05-10', 'Tue', 3),
(117, 1, '2016-05-11', 'Wed', 4),
(118, 1, '2016-05-12', 'Thu', 5),
(119, 1, '2016-05-13', 'Fri', 6),

-- a partial end of the week, the whole week, partial start of the week (without weekends)
(223, 2, '2016-05-05', 'Thu', 5),
(224, 2, '2016-05-06', 'Fri', 6),
(225, 2, '2016-05-09', 'Mon', 2),
(226, 2, '2016-05-10', 'Tue', 3),
(227, 2, '2016-05-11', 'Wed', 4),
(228, 2, '2016-05-12', 'Thu', 5),
(229, 2, '2016-05-13', 'Fri', 6),
(230, 2, '2016-05-16', 'Mon', 2),
(231, 2, '2016-05-17', 'Tue', 3),

-- only Mon, Wed, Fri are included across two weeks plus partial third week
(310, 3, '2016-05-02', 'Mon', 2),
(311, 3, '2016-05-04', 'Wed', 4),
(314, 3, '2016-05-06', 'Fri', 6),
(315, 3, '2016-05-09', 'Mon', 2),
(317, 3, '2016-05-11', 'Wed', 4),
(319, 3, '2016-05-13', 'Fri', 6),
(330, 3, '2016-05-16', 'Mon', 2),

-- a whole week (without weekend), in the second week Mon is not included
(410, 4, '2016-05-02', 'Mon', 2),
(411, 4, '2016-05-03', 'Tue', 3),
(412, 4, '2016-05-04', 'Wed', 4),
(413, 4, '2016-05-05', 'Thu', 5),
(414, 4, '2016-05-06', 'Fri', 6),
(416, 4, '2016-05-10', 'Tue', 3),
(417, 4, '2016-05-11', 'Wed', 4),
(418, 4, '2016-05-12', 'Thu', 5),
(419, 4, '2016-05-13', 'Fri', 6),

-- three weeks, but without Mon in the second week (no weekends)
(510, 5, '2016-05-02', 'Mon', 2),
(511, 5, '2016-05-03', 'Tue', 3),
(512, 5, '2016-05-04', 'Wed', 4),
(513, 5, '2016-05-05', 'Thu', 5),
(514, 5, '2016-05-06', 'Fri', 6),
(516, 5, '2016-05-10', 'Tue', 3),
(517, 5, '2016-05-11', 'Wed', 4),
(518, 5, '2016-05-12', 'Thu', 5),
(519, 5, '2016-05-13', 'Fri', 6),
(520, 5, '2016-05-16', 'Mon', 2),
(521, 5, '2016-05-17', 'Tue', 3),
(522, 5, '2016-05-18', 'Wed', 4),
(523, 5, '2016-05-19', 'Thu', 5),
(524, 5, '2016-05-20', 'Fri', 6),

-- long gap between two intervals
(623, 6, '2016-05-05', 'Thu', 5),
(624, 6, '2016-05-06', 'Fri', 6),
(625, 6, '2016-05-09', 'Mon', 2),
(626, 6, '2016-05-10', 'Tue', 3),
(627, 6, '2016-05-11', 'Wed', 4),
(628, 6, '2016-05-12', 'Thu', 5),
(629, 6, '2016-05-13', 'Fri', 6),
(630, 6, '2016-05-16', 'Mon', 2),
(631, 6, '2016-05-17', 'Tue', 3),
(645, 6, '2016-06-06', 'Mon', 2),
(646, 6, '2016-06-07', 'Tue', 3),
(647, 6, '2016-06-08', 'Wed', 4),
(648, 6, '2016-06-09', 'Thu', 5),
(649, 6, '2016-06-10', 'Fri', 6),
(655, 6, '2016-06-13', 'Mon', 2),
(656, 6, '2016-06-14', 'Tue', 3),
(657, 6, '2016-06-15', 'Wed', 4),
(658, 6, '2016-06-16', 'Thu', 5),
(659, 6, '2016-06-17', 'Fri', 6),

-- two weeks, no gaps between days at all, even weekends are included
(710, 7, '2016-05-02', 'Mon', 2),
(711, 7, '2016-05-03', 'Tue', 3),
(712, 7, '2016-05-04', 'Wed', 4),
(713, 7, '2016-05-05', 'Thu', 5),
(714, 7, '2016-05-06', 'Fri', 6),
(715, 7, '2016-05-07', 'Sat', 7),
(716, 7, '2016-05-08', 'Sun', 1),
(725, 7, '2016-05-09', 'Mon', 2),
(726, 7, '2016-05-10', 'Tue', 3),
(727, 7, '2016-05-11', 'Wed', 4),
(728, 7, '2016-05-12', 'Thu', 5),
(729, 7, '2016-05-13', 'Fri', 6),

-- no gaps between days at all, even weekends are included, with partial weeks
(805, 8, '2016-04-30', 'Sat', 7),
(806, 8, '2016-05-01', 'Sun', 1),
(810, 8, '2016-05-02', 'Mon', 2),
(811, 8, '2016-05-03', 'Tue', 3),
(812, 8, '2016-05-04', 'Wed', 4),
(813, 8, '2016-05-05', 'Thu', 5),
(814, 8, '2016-05-06', 'Fri', 6),
(815, 8, '2016-05-07', 'Sat', 7),
(816, 8, '2016-05-08', 'Sun', 1),
(825, 8, '2016-05-09', 'Mon', 2),
(826, 8, '2016-05-10', 'Tue', 3),
(827, 8, '2016-05-11', 'Wed', 4),
(828, 8, '2016-05-12', 'Thu', 5),
(829, 8, '2016-05-13', 'Fri', 6),
(830, 8, '2016-05-14', 'Sat', 7),

-- only Mon-Wed included, two weeks plus partial third week
(910, 9, '2016-05-02', 'Mon', 2),
(911, 9, '2016-05-03', 'Tue', 3),
(912, 9, '2016-05-04', 'Wed', 4),
(915, 9, '2016-05-09', 'Mon', 2),
(916, 9, '2016-05-10', 'Tue', 3),
(917, 9, '2016-05-11', 'Wed', 4),
(930, 9, '2016-05-16', 'Mon', 2),
(931, 9, '2016-05-17', 'Tue', 3),

-- only Thu-Sun included, three weeks
(1013,10,'2016-05-05', 'Thu', 5),
(1014,10,'2016-05-06', 'Fri', 6),
(1015,10,'2016-05-07', 'Sat', 7),
(1016,10,'2016-05-08', 'Sun', 1),
(1018,10,'2016-05-12', 'Thu', 5),
(1019,10,'2016-05-13', 'Fri', 6),
(1020,10,'2016-05-14', 'Sat', 7),
(1021,10,'2016-05-15', 'Sun', 1),
(1023,10,'2016-05-19', 'Thu', 5),
(1024,10,'2016-05-20', 'Fri', 6),
(1025,10,'2016-05-21', 'Sat', 7),
(1026,10,'2016-05-22', 'Sun', 1),

-- only Tue for first three weeks, then only Thu for the next three weeks
(1111,11,'2016-05-03', 'Tue', 3),
(1116,11,'2016-05-10', 'Tue', 3),
(1131,11,'2016-05-17', 'Tue', 3),
(1123,11,'2016-05-19', 'Thu', 5),
(1124,11,'2016-05-26', 'Thu', 5),
(1125,11,'2016-06-02', 'Thu', 5),

-- one week, then one week gap, then one week
(1210,12,'2016-05-02', 'Mon', 2),
(1211,12,'2016-05-03', 'Tue', 3),
(1212,12,'2016-05-04', 'Wed', 4),
(1213,12,'2016-05-05', 'Thu', 5),
(1214,12,'2016-05-06', 'Fri', 6),
(1215,12,'2016-05-16', 'Mon', 2),
(1216,12,'2016-05-17', 'Tue', 3),
(1217,12,'2016-05-18', 'Wed', 4),
(1218,12,'2016-05-19', 'Thu', 5),
(1219,12,'2016-05-20', 'Fri', 6);

SELECT ID, ContractID, dt, dowChar, dowInt
FROM @Src
ORDER BY ContractID, dt;


DECLARE @Dst TABLE (ContractID int, StartDT date, EndDT date, DayCount int, WeekDays varchar(255));
INSERT INTO @Dst (ContractID, StartDT, EndDT, DayCount, WeekDays) VALUES
(1, '2016-05-02', '2016-05-13', 10, 'Mon,Tue,Wed,Thu,Fri,'),
(2, '2016-05-05', '2016-05-17',  9, 'Mon,Tue,Wed,Thu,Fri,'),
(3, '2016-05-02', '2016-05-16',  7, 'Mon,Wed,Fri,'),
(4, '2016-05-02', '2016-05-06',  5, 'Mon,Tue,Wed,Thu,Fri,'),
(4, '2016-05-10', '2016-05-13',  4, 'Tue,Wed,Thu,Fri,'),
(5, '2016-05-02', '2016-05-06',  5, 'Mon,Tue,Wed,Thu,Fri,'),
(5, '2016-05-10', '2016-05-20',  9, 'Mon,Tue,Wed,Thu,Fri,'),
(6, '2016-05-05', '2016-05-17',  9, 'Mon,Tue,Wed,Thu,Fri,'),
(6, '2016-06-06', '2016-06-17', 10, 'Mon,Tue,Wed,Thu,Fri,'),
(7, '2016-05-02', '2016-05-13', 12, 'Sun,Mon,Tue,Wed,Thu,Fri,Sat,'),
(8, '2016-04-30', '2016-05-14', 15, 'Sun,Mon,Tue,Wed,Thu,Fri,Sat,'),
(9, '2016-05-02', '2016-05-17',  8, 'Mon,Tue,Wed,'),
(10,'2016-05-05', '2016-05-22', 12, 'Sun,Thu,Fri,Sat,'),
(11,'2016-05-03', '2016-05-17',  3, 'Tue,'),
(11,'2016-05-19', '2016-06-02',  3, 'Thu,'),
(12,'2016-05-02', '2016-05-06',  5, 'Mon,Tue,Wed,Thu,Fri,'),
(12,'2016-05-16', '2016-05-20',  5, 'Mon,Tue,Wed,Thu,Fri,');

SELECT ContractID, StartDT, EndDT, DayCount, WeekDays
FROM @Dst
ORDER BY ContractID, StartDT;

Comparison of answers

The real table @Src has 403,555 rows with 15,857 distinct ContractIDs. All answers produce correct results (at least for my data) and all of them are reasonably fast, but they differ in optimality. The less intervals generated, the better. I included run times just for curiosity. The main focus is the correct and optimal result, not the speed (unless it takes too long - I stopped the non-recursive query by Ziggy Crueltyfree Zeitgeister after 10 minutes).

+--------------------------------------------------------+-----------+---------+
|                         Answer                         | Intervals | Seconds |
+--------------------------------------------------------+-----------+---------+
| Ziggy Crueltyfree Zeitgeister                          |     25751 |    7.88 |
| While loop                                             |           |         |
|                                                        |           |         |
| Ziggy Crueltyfree Zeitgeister                          |     25751 |    8.27 |
| Recursive                                              |           |         |
|                                                        |           |         |
| Michael Green                                          |     25751 |   22.63 |
| Recursive                                              |           |         |
|                                                        |           |         |
| Geoff Patterson                                        |     26670 |    4.79 |
| Weekly gaps-and-islands with merging of partial weeks  |           |         |
|                                                        |           |         |
| Vladimir Baranov                                       |     34560 |    4.03 |
| Daily, then weekly gaps-and-islands                    |           |         |
|                                                        |           |         |
| Mikael Eriksson                                        |     35840 |    0.65 |
| Weekly gaps-and-islands                                |           |         |
+--------------------------------------------------------+-----------+---------+
| Vladimir Baranov                                       |     25751 |  121.51 |
| Cursor                                                 |           |         |
+--------------------------------------------------------+-----------+---------+

Answer 1

This one uses a recursive CTE. Its result is identical to the example in the question. It was a nightmare to come up with... The code includes comments to ease through its convoluted logic.

SET DATEFIRST 1 -- Make Monday weekday=1

DECLARE @Ranked TABLE (RowID int NOT NULL IDENTITY PRIMARY KEY,                   -- Incremental uninterrupted sequence in the right order
                       ID int NOT NULL UNIQUE, ContractID int NOT NULL, dt date,  -- Original relevant values (ID is not really necessary)
                       WeekNo int NOT NULL, dowBit int NOT NULL);                 -- Useful to find gaps in days or weeks
INSERT INTO @Ranked
SELECT ID, ContractID, dt,
       DATEDIFF(WEEK, '1900-01-01', DATEADD(DAY, 1-DATEPART(dw, dt), dt)) AS WeekNo,
       POWER(2, DATEPART(dw, dt)-1) AS dowBit
FROM @Src
ORDER BY ContractID, WeekNo, dowBit

/*
Each evaluated date makes part of the carried sequence if:
  - this is not a new contract, and
    - sequence started this week, or
    - same day last week was part of the sequence, or
    - sequence started last week and today is a lower day than the accumulated weekdays list
  - and there are no sequence gaps since previous day
(otherwise it does not make part of the old sequence, so it starts a new one) */

DECLARE @RankedRanges TABLE (RowID int NOT NULL PRIMARY KEY, WeekDays int NOT NULL, StartRowID int NULL);

WITH WeeksCTE AS -- Needed for building the sequence gradually, and comparing the carried sequence (and previous day) with a current evaluated day
( 
    SELECT RowID, ContractID, dowBit, WeekNo, RowID AS StartRowID, WeekNo AS StartWN, dowBit AS WeekDays, dowBit AS StartWeekDays
    FROM @Ranked
    WHERE RowID = 1 
    UNION ALL
    SELECT RowID, ContractID, dowBit, WeekNo, StartRowID,
           CASE WHEN StartRowID IS NULL THEN StartWN ELSE WeekNo END AS WeekNo,
           CASE WHEN StartRowID IS NULL THEN WeekDays | dowBit ELSE dowBit END AS WeekDays,
           CASE WHEN StartRowID IS NOT NULL THEN dowBit WHEN WeekNo = StartWN THEN StartWeekDays | dowBit ELSE StartWeekDays END AS StartWeekDays
    FROM (
        SELECT w.*, pre.StartWN, pre.WeekDays, pre.StartWeekDays,
               CASE WHEN w.ContractID <> pre.ContractID OR     -- New contract always break the sequence
                         NOT (w.WeekNo = pre.StartWN OR        -- Same week as a new sequence always keeps the sequence
                              w.dowBit & pre.WeekDays > 0 OR   -- Days in the sequence keep the sequence (provided there are no gaps, checked later)
                              (w.WeekNo = pre.StartWN+1 AND (w.dowBit-1) & pre.StartWeekDays = 0)) OR -- Days in the second week when less than a week passed since the sequence started remain in sequence
                         (w.WeekNo > pre.StartWN AND -- look for gap after initial week
                          w.WeekNo > pre.WeekNo+1 OR -- look for full-week gaps
                          (w.WeekNo = pre.WeekNo AND                            -- when same week as previous day,
                           ((w.dowBit-1) ^ (pre.dowBit*2-1)) & pre.WeekDays > 0 -- days between this and previous weekdays, compared to current series
                          ) OR
                          (w.WeekNo > pre.WeekNo AND                                   -- when following week of previous day,
                           ((-1 ^ (pre.dowBit*2-1)) | (w.dowBit-1)) & pre.WeekDays > 0 -- days between this and previous weekdays, compared to current series
                          )) THEN w.RowID END AS StartRowID
        FROM WeeksCTE pre
        JOIN @Ranked w ON (w.RowID = pre.RowID + 1)
        ) w
) 
INSERT INTO @RankedRanges -- days sequence and starting point of each sequence
SELECT RowID, WeekDays, StartRowID
--SELECT *
FROM WeeksCTE
OPTION (MAXRECURSION 0)

--SELECT * FROM @RankedRanges

DECLARE @Ranges TABLE (RowNo int NOT NULL IDENTITY PRIMARY KEY, RowID int NOT NULL);

INSERT INTO @Ranges       -- @RankedRanges filtered only by start of each range, with numbered rows to easily find the end of each range
SELECT StartRowID
FROM @RankedRanges
WHERE StartRowID IS NOT NULL
ORDER BY 1

-- Final result putting everything together
SELECT rs.ContractID, rs.dt AS StartDT, re.dt AS EndDT, re.RowID-rs.RowID+1 AS DayCount,
       CASE WHEN rr.WeekDays & 64 > 0 THEN 'Sun,' ELSE '' END +
       CASE WHEN rr.WeekDays & 1 > 0 THEN 'Mon,' ELSE '' END +
       CASE WHEN rr.WeekDays & 2 > 0 THEN 'Tue,' ELSE '' END +
       CASE WHEN rr.WeekDays & 4 > 0 THEN 'Wed,' ELSE '' END +
       CASE WHEN rr.WeekDays & 8 > 0 THEN 'Thu,' ELSE '' END +
       CASE WHEN rr.WeekDays & 16 > 0 THEN 'Fri,' ELSE '' END +
       CASE WHEN rr.WeekDays & 32 > 0 THEN 'Sat,' ELSE '' END AS WeekDays
FROM (
    SELECT r.RowID AS StartRowID, COALESCE(pos.RowID-1, (SELECT MAX(RowID) FROM @Ranked)) AS EndRowID
    FROM @Ranges r
    LEFT JOIN @Ranges pos ON (pos.RowNo = r.RowNo + 1)
    ) g
JOIN @Ranked rs ON (rs.RowID = g.StartRowID)
JOIN @Ranked re ON (re.RowID = g.EndRowID)
JOIN @RankedRanges rr ON (rr.RowID = re.RowID)

---

Another strategy

This one should be significantly faster than the previous one because it doesn't rely on the slow limited recursive CTE in SQL Server 2008, although it implements more or less the same strategy.

There is a WHILE loop (I couldn't devise a way to avoid it), but goes for a reduced number of iterations (the highest number of sequences (minus one) on any given contract).

It's a simple strategy, and could be used for sequences either shorter or longer than a week (replacing any occurrence of the constant 7 for any other number, and the dowBit calculated from MODULUS x of DayNo rather than DATEPART(wk)) and up to 32.

SET DATEFIRST 1 -- Make Monday weekday=1

-- Get the minimum information needed to calculate sequences
DECLARE @Days TABLE (ContractID int NOT NULL, dt date, DayNo int NOT NULL, dowBit int NOT NULL, PRIMARY KEY (ContractID, DayNo));
INSERT INTO @Days
SELECT ContractID, dt, CAST(CAST(dt AS datetime) AS int) AS DayNo, POWER(2, DATEPART(dw, dt)-1) AS dowBit
FROM @Src

DECLARE @RangeStartFirstPass TABLE (ContractID int NOT NULL, DayNo int NOT NULL, PRIMARY KEY (ContractID, DayNo))

-- Calculate, from the above list, which days are not present in the previous 7
INSERT INTO @RangeStartFirstPass
SELECT r.ContractID, r.DayNo
FROM @Days r
LEFT JOIN @Days pr ON (pr.ContractID = r.ContractID AND pr.DayNo BETWEEN r.DayNo-7 AND r.DayNo-1) -- Last 7 days
GROUP BY r.ContractID, r.DayNo, r.dowBit
HAVING r.dowBit & COALESCE(SUM(pr.dowBit), 0) = 0

-- Update the previous list with all days that occur right after a missing day
INSERT INTO @RangeStartFirstPass
SELECT *
FROM (
    SELECT DISTINCT ContractID, (SELECT MIN(DayNo) FROM @Days WHERE ContractID = d.ContractID AND DayNo > d.DayNo + 7) AS DayNo
    FROM @Days d
    WHERE NOT EXISTS (SELECT 1 FROM @Days WHERE ContractID = d.ContractID AND DayNo = d.DayNo + 7)
    ) d
WHERE DayNo IS NOT NULL AND
      NOT EXISTS (SELECT 1 FROM @RangeStartFirstPass WHERE ContractID = d.ContractID AND DayNo = d.DayNo)

DECLARE @RangeStart TABLE (ContractID int NOT NULL, DayNo int NOT NULL, PRIMARY KEY (ContractID, DayNo));

-- Fetch the first sequence for each contract
INSERT INTO @RangeStart
SELECT ContractID, MIN(DayNo)
FROM @RangeStartFirstPass
GROUP BY ContractID

-- Add to the list above the next sequence for each contract, until all are added
-- (ensure no sequence is added with less than 7 days)
WHILE @@ROWCOUNT > 0
  INSERT INTO @RangeStart
  SELECT f.ContractID, MIN(f.DayNo)
  FROM (SELECT ContractID, MAX(DayNo) AS DayNo FROM @RangeStart GROUP BY ContractID) s
  JOIN @RangeStartFirstPass f ON (f.ContractID = s.ContractID AND f.DayNo > s.DayNo + 7)
  GROUP BY f.ContractID

-- Summarise results
SELECT ContractID, StartDT, EndDT, DayCount,
       CASE WHEN WeekDays & 64 > 0 THEN 'Sun,' ELSE '' END +
       CASE WHEN WeekDays & 1 > 0 THEN 'Mon,' ELSE '' END +
       CASE WHEN WeekDays & 2 > 0 THEN 'Tue,' ELSE '' END +
       CASE WHEN WeekDays & 4 > 0 THEN 'Wed,' ELSE '' END +
       CASE WHEN WeekDays & 8 > 0 THEN 'Thu,' ELSE '' END +
       CASE WHEN WeekDays & 16 > 0 THEN 'Fri,' ELSE '' END +
       CASE WHEN WeekDays & 32 > 0 THEN 'Sat,' ELSE '' END AS WeekDays
FROM (
    SELECT r.ContractID,
           MIN(d.dt) AS StartDT,
           MAX(d.dt) AS EndDT,
           COUNT(*) AS DayCount,
           SUM(DISTINCT d.dowBit) AS WeekDays
    FROM (SELECT *, COALESCE((SELECT MIN(DayNo) FROM @RangeStart WHERE ContractID = rs.ContractID AND DayNo > rs.DayNo), 999999) AS DayEnd FROM @RangeStart rs) r
    JOIN @Days d ON (d.ContractID = r.ContractID AND d.DayNo BETWEEN r.DayNo AND r.DayEnd-1)
    GROUP BY r.ContractID, r.DayNo
    ) d
ORDER BY ContractID, StartDT

Answer 2

Not exactly what you are looking for but could perhaps be of interest to you.

The query creates weeks with a comma separated string for the days used in each week. It then finds the islands of consecutive weeks that uses the same pattern in Weekdays.

with Weeks as
(
  select T.*,
         row_number() over(partition by T.ContractID, T.WeekDays order by T.WeekNumber) as rn
  from (
       select S1.ContractID,
              min(S1.dt) as StartDT,
              max(S1.dt) as EndDT,
              datediff(day, 0, S1.dt) / 7 as WeekNumber, -- Number of weeks since '1900-01-01 (a monday)'
              count(*) as DayCount,
              stuff((
                    select ','+S2.dowChar
                    from @Src as S2
                    where S2.ContractID = S1.ContractID and
                          S2.dt between min(S1.dt) and max(S1.dt)
                    order by S2.dt
                    for xml path('')
                    ), 1, 1, '') as WeekDays
       from @Src as S1
       group by S1.ContractID, 
                datediff(day, 0, S1.dt) / 7
       ) as T
)
select W.ContractID,
       min(W.StartDT) as StartDT,
       max(W.EndDT) as EndDT,
       count(*) * W.DayCount as DayCount,
       W.WeekDays
from Weeks as W
group by W.ContractID,
         W.WeekDays,
         W.DayCount,
         W.rn - W.WeekNumber
order by W.ContractID,
         min(W.WeekNumber);

Result:

ContractID  StartDT    EndDT      DayCount    WeekDays
----------- ---------- ---------- ----------- -----------------------------
1           2016-05-02 2016-05-13 10          Mon,Tue,Wed,Thu,Fri
2           2016-05-05 2016-05-06 2           Thu,Fri
2           2016-05-09 2016-05-13 5           Mon,Tue,Wed,Thu,Fri
2           2016-05-16 2016-05-17 2           Mon,Tue
3           2016-05-02 2016-05-13 6           Mon,Wed,Fri
3           2016-05-16 2016-05-16 1           Mon
4           2016-05-02 2016-05-06 5           Mon,Tue,Wed,Thu,Fri
4           2016-05-10 2016-05-13 4           Tue,Wed,Thu,Fri
5           2016-05-02 2016-05-06 5           Mon,Tue,Wed,Thu,Fri
5           2016-05-10 2016-05-13 4           Tue,Wed,Thu,Fri
5           2016-05-16 2016-05-20 5           Mon,Tue,Wed,Thu,Fri
6           2016-05-05 2016-05-06 2           Thu,Fri
6           2016-05-09 2016-05-13 5           Mon,Tue,Wed,Thu,Fri
6           2016-05-16 2016-05-17 2           Mon,Tue
6           2016-06-06 2016-06-17 10          Mon,Tue,Wed,Thu,Fri
7           2016-05-02 2016-05-08 7           Mon,Tue,Wed,Thu,Fri,Sat,Sun
7           2016-05-09 2016-05-13 5           Mon,Tue,Wed,Thu,Fri
8           2016-04-30 2016-05-01 2           Sat,Sun
8           2016-05-02 2016-05-08 7           Mon,Tue,Wed,Thu,Fri,Sat,Sun
8           2016-05-09 2016-05-14 6           Mon,Tue,Wed,Thu,Fri,Sat
9           2016-05-02 2016-05-11 6           Mon,Tue,Wed
9           2016-05-16 2016-05-17 2           Mon,Tue
10          2016-05-05 2016-05-22 12          Thu,Fri,Sat,Sun
11          2016-05-03 2016-05-10 2           Tue
11          2016-05-17 2016-05-19 2           Tue,Thu
11          2016-05-26 2016-06-02 2           Thu

ContractID = 2 shows what the difference in the result is compared to what you want. The first and last week will be treated as separate periods since WeekDays is different.

Answer 3

I ended up with an approach that yields the optimal solution in this case and I think will do well in general. The solution is quite lengthy, however, so it would be interesting to see if someone else has a different approach that is more concise.

Here is a script that contains the full solution.

And here is an outline of the algorithm:

• Pivot the data set so that there is a single row representing each week
• Compute the islands of weeks within each ContractId
• Merge any adjacent weeks that fall within the same ContractId and have same WeekDays
• For any single weeks (not yet merged) where the previous grouping is on the same island and the WeekDays of the single week matches a leading subset of the WeekDays of the previous grouping, merge into that previous grouping
• For any single weeks (not yet merged) where the next grouping is on the same island and the WeekDays of the single week matches a trailing subset of the WeekDays of the next grouping, merge into that next grouping
• For any two adjacent weeks on the same island where neither has been merged, merge them together if they are both partial weeks that can be combined (e.g., "Mon,Tue,Wed,Thu," and "Wed,Thu,Sat,")
• For any remaining single weeks (not yet merged), if possible split the week into two parts and merge both parts, the first part into the previous grouping on the same island, and the second part into the following grouping on the same island

Answer 4

I couldn't understand the logic behind grouping weeks with gaps, or weeks with weekends (e.g. when there are two consecutive weeks with a weekend, which week does the weekend go to?).

The following query produces the desired output except that it only groups consecutive weekdays, and groups weeks Sun-Sat (rather than Mon-Sun). Whilst not exactly what you want, maybe this is can provide some clues for a different strategy. The grouping of days comes from here. The windowing functions used should work with SQLServer 2008, but I don't have that version to test if it actually does.

WITH 
  mysrc AS (
    SELECT *, RANK() OVER (PARTITION BY ContractID ORDER BY DT) AS rank
    FROM @Src
    ),
  prepos AS (
    SELECT s.*, pos.ID AS posid
    FROM mysrc s
    LEFT JOIN mysrc pos ON (pos.ContractID = s.ContractID AND pos.rank = s.rank+1 AND (pos.DowInt = s.DowInt+1 OR pos.DowInt = 2 AND s.DowInt=6))
    ),
  grped AS (
    SELECT TOP 100 *, (SELECT COUNT(CASE WHEN posid IS NULL THEN 1 END) FROM prepos WHERE contractid = p.contractid AND rank < p.rank) as grp
    FROM prepos p
    ORDER BY ContractID, DT
    )
SELECT ContractID, min(dt) AS StartDT, max(dt) AS EndDT, count(*) AS DayCount,
       STUFF( (SELECT ', ' + dowchar
               FROM (
                 SELECT TOP 100 dowint, dowchar 
                 FROM grped 
                 WHERE ContractID = g.ContractID AND grp = g.grp 
                 GROUP BY dowint, dowchar 
                 ORDER BY 1
                 ) a 
               FOR XML PATH(''), TYPE).value('.','varchar(max)'), 1, 2, '') AS WeekDays
FROM grped g
GROUP BY ContractID, grp
ORDER BY 1, 2

Result

+------------+------------+------------+----------+-----------------------------------+
| ContractID | StartDT    | EndDT      | DayCount | WeekDays                          |
+------------+------------+------------+----------+-----------------------------------+
| 1          | 2/05/2016  | 13/05/2016 | 10       | Mon, Tue, Wed, Thu, Fri           |
| 2          | 5/05/2016  | 17/05/2016 | 9        | Mon, Tue, Wed, Thu, Fri           |
| 3          | 2/05/2016  | 2/05/2016  | 1        | Mon                               |
| 3          | 4/05/2016  | 4/05/2016  | 1        | Wed                               |
| 3          | 6/05/2016  | 9/05/2016  | 2        | Mon, Fri                          |
| 3          | 11/05/2016 | 11/05/2016 | 1        | Wed                               |
| 3          | 13/05/2016 | 16/05/2016 | 2        | Mon, Fri                          |
| 4          | 2/05/2016  | 6/05/2016  | 5        | Mon, Tue, Wed, Thu, Fri           |
| 4          | 10/05/2016 | 13/05/2016 | 4        | Tue, Wed, Thu, Fri                |
| 5          | 2/05/2016  | 6/05/2016  | 5        | Mon, Tue, Wed, Thu, Fri           |
| 5          | 10/05/2016 | 20/05/2016 | 9        | Mon, Tue, Wed, Thu, Fri           |
| 6          | 5/05/2016  | 17/05/2016 | 9        | Mon, Tue, Wed, Thu, Fri           |
| 6          | 6/06/2016  | 17/06/2016 | 10       | Mon, Tue, Wed, Thu, Fri           |
| 7          | 2/05/2016  | 7/05/2016  | 6        | Mon, Tue, Wed, Thu, Fri, Sat      |
| 7          | 8/05/2016  | 13/05/2016 | 6        | Sun, Mon, Tue, Wed, Thu, Fri      |
| 8          | 30/04/2016 | 30/04/2016 | 1        | Sat                               |
| 8          | 1/05/2016  | 7/05/2016  | 7        | Sun, Mon, Tue, Wed, Thu, Fri, Sat |
| 8          | 8/05/2016  | 14/05/2016 | 7        | Sun, Mon, Tue, Wed, Thu, Fri, Sat |
| 9          | 2/05/2016  | 4/05/2016  | 3        | Mon, Tue, Wed                     |
| 9          | 9/05/2016  | 10/05/2016 | 2        | Mon, Tue                          |
+------------+------------+------------+----------+-----------------------------------+

Answer 5

For the sake of completeness, here is a two-pass gaps-and-islands approach that I tried myself before asking this question.

As I was testing it on the real data I found few cases when it was producing incorrect results and fixed it.

Here is the algorithm:

• Generate islands of consecutive dates (CTE_ContractDays, CTE_DailyRN, CTE_DailyIslands) and calculate a week number for each starting and ending date of an island. Here week number is calculated assuming that Monday is the first day of the week.
• If the schedule has non-sequential dates within the same week (like in example 3), the previous stage will create several rows for the same week. Group rows to have only one row per week (CTE_Weeks).
• For each row from the previous stage build a comma-separated list of week days (CTE_FirstResult).
• Second pass of gaps-and-islands to group consecutive weeks with the same WeekDays (CTE_SecondRN, CTE_Schedules).

It handles well cases when there is no disruption in the weekly patterns (1, 7, 8, 10, 12). It handles well cases when pattern has non-sequential days (3).

But, unfortunately, it generates extra intervals for partial weeks (2, 3, 5, 6, 9, 11).

WITH
CTE_ContractDays
AS
(
    SELECT
         S.ContractID
        ,MIN(S.dt) OVER (PARTITION BY S.ContractID) AS ContractMinDT
        ,S.dt
        ,ROW_NUMBER() OVER (PARTITION BY S.ContractID ORDER BY S.dt) AS rn1
        ,DATEDIFF(day, '2001-01-01', S.dt) AS DayNumber
        ,S.dowChar
        ,S.dowInt
    FROM
        @Src AS S
)
,CTE_DailyRN
AS
(
    SELECT
        DayNumber - rn1 AS WeekGroupNumber
        ,ROW_NUMBER() OVER (
            PARTITION BY
                ContractID
                ,DayNumber - rn1
            ORDER BY dt) AS rn2
        ,ContractID
        ,ContractMinDT
        ,dt
        ,rn1
        ,DayNumber
        ,dowChar
        ,dowInt
    FROM CTE_ContractDays
)
,CTE_DailyIslands
AS
(
    SELECT
        ContractID
        ,ContractMinDT
        ,MIN(dt) AS MinDT
        ,MAX(dt) AS MaxDT
        ,COUNT(*) AS DayCount
        -- '2001-01-01' is Monday
        ,DATEDIFF(day, '2001-01-01', MIN(dt)) / 7 AS WeekNumberMin
        ,DATEDIFF(day, '2001-01-01', MAX(dt)) / 7 AS WeekNumberMax
    FROM CTE_DailyRN
    GROUP BY
        ContractID
        ,rn1-rn2
        ,ContractMinDT
)
,CTE_Weeks
AS
(
    SELECT
        ContractID
        ,ContractMinDT
        ,MIN(MinDT) AS MinDT
        ,MAX(MaxDT) AS MaxDT
        ,SUM(DayCount) AS DayCount
        ,WeekNumberMin
        ,WeekNumberMax
    FROM CTE_DailyIslands
    GROUP BY
        ContractID
        ,ContractMinDT
        ,WeekNumberMin
        ,WeekNumberMax
)
,CTE_FirstResult
AS
(
    SELECT
        ContractID
        ,ContractMinDT
        ,MinDT
        ,MaxDT
        ,DayCount
        ,CA_Data.XML_Value AS DaysOfWeek
        ,WeekNumberMin AS WeekNumber
        ,ROW_NUMBER() OVER(PARTITION BY ContractID ORDER BY MinDT) AS rn1
    FROM
        CTE_Weeks
        CROSS APPLY
        (
            SELECT CAST(CTE_ContractDays.dowChar AS varchar(8000)) + ',' AS dw
            FROM CTE_ContractDays
            WHERE
                    CTE_ContractDays.ContractID = CTE_Weeks.ContractID
                AND CTE_ContractDays.dt >= CTE_Weeks.MinDT
                AND CTE_ContractDays.dt <= CTE_Weeks.MaxDT
            GROUP BY
                CTE_ContractDays.dowChar
                ,CTE_ContractDays.dowInt
            ORDER BY CTE_ContractDays.dowInt
            FOR XML PATH(''), TYPE
        ) AS CA_XML(XML_Value)
        CROSS APPLY
        (
            SELECT CA_XML.XML_Value.value('.', 'VARCHAR(8000)')
        ) AS CA_Data(XML_Value)
)
,CTE_SecondRN
AS
(
    SELECT 
        ContractID
        ,ContractMinDT
        ,MinDT
        ,MaxDT
        ,DayCount
        ,DaysOfWeek
        ,WeekNumber
        ,rn1
        ,WeekNumber - rn1 AS SecondGroupNumber
        ,ROW_NUMBER() OVER (
            PARTITION BY
                ContractID
                ,DaysOfWeek
                ,DayCount
                ,WeekNumber - rn1
            ORDER BY MinDT) AS rn2
    FROM CTE_FirstResult
)
,CTE_Schedules
AS
(
    SELECT
        ContractID
        ,MIN(MinDT) AS StartDT
        ,MAX(MaxDT) AS EndDT
        ,SUM(DayCount) AS DayCount
        ,DaysOfWeek
    FROM CTE_SecondRN
    GROUP BY
        ContractID
        ,DaysOfWeek
        ,rn1-rn2
)
SELECT
    ContractID
    ,StartDT
    ,EndDT
    ,DayCount
    ,DaysOfWeek AS WeekDays
FROM CTE_Schedules
ORDER BY
    ContractID
    ,StartDT
;

Result

+------------+------------+------------+----------+------------------------------+
| ContractID |  StartDT   |   EndDT    | DayCount |           WeekDays           |
+------------+------------+------------+----------+------------------------------+
|          1 | 2016-05-02 | 2016-05-13 |       10 | Mon,Tue,Wed,Thu,Fri,         |
|          2 | 2016-05-05 | 2016-05-06 |        2 | Thu,Fri,                     |
|          2 | 2016-05-09 | 2016-05-13 |        5 | Mon,Tue,Wed,Thu,Fri,         |
|          2 | 2016-05-16 | 2016-05-17 |        2 | Mon,Tue,                     |
|          3 | 2016-05-02 | 2016-05-13 |        6 | Mon,Wed,Fri,                 |
|          3 | 2016-05-16 | 2016-05-16 |        1 | Mon,                         |
|          4 | 2016-05-02 | 2016-05-06 |        5 | Mon,Tue,Wed,Thu,Fri,         |
|          4 | 2016-05-10 | 2016-05-13 |        4 | Tue,Wed,Thu,Fri,             |
|          5 | 2016-05-02 | 2016-05-06 |        5 | Mon,Tue,Wed,Thu,Fri,         |
|          5 | 2016-05-10 | 2016-05-13 |        4 | Tue,Wed,Thu,Fri,             |
|          5 | 2016-05-16 | 2016-05-20 |        5 | Mon,Tue,Wed,Thu,Fri,         |
|          6 | 2016-05-05 | 2016-05-06 |        2 | Thu,Fri,                     |
|          6 | 2016-05-09 | 2016-05-13 |        5 | Mon,Tue,Wed,Thu,Fri,         |
|          6 | 2016-05-16 | 2016-05-17 |        2 | Mon,Tue,                     |
|          6 | 2016-06-06 | 2016-06-17 |       10 | Mon,Tue,Wed,Thu,Fri,         |
|          7 | 2016-05-02 | 2016-05-13 |       12 | Sun,Mon,Tue,Wed,Thu,Fri,Sat, |
|          8 | 2016-04-30 | 2016-05-14 |       15 | Sun,Mon,Tue,Wed,Thu,Fri,Sat, |
|          9 | 2016-05-02 | 2016-05-11 |        6 | Mon,Tue,Wed,                 |
|          9 | 2016-05-16 | 2016-05-17 |        2 | Mon,Tue,                     |
|         10 | 2016-05-05 | 2016-05-22 |       12 | Sun,Thu,Fri,Sat,             |
|         11 | 2016-05-03 | 2016-05-10 |        2 | Tue,                         |
|         11 | 2016-05-17 | 2016-05-19 |        2 | Tue,Thu,                     |
|         11 | 2016-05-26 | 2016-06-02 |        2 | Thu,                         |
|         12 | 2016-05-02 | 2016-05-06 |        5 | Mon,Tue,Wed,Thu,Fri,         |
|         12 | 2016-05-16 | 2016-05-20 |        5 | Mon,Tue,Wed,Thu,Fri,         |
+------------+------------+------------+----------+------------------------------+

---

Cursor-based solution

I converted my C# code into a cursor-based algorithm, just to see how it compares to other solutions on real data. It confirms that it is much slower than other set-based or recursive approaches, but it generates an optimal result.

CREATE TABLE #Dst_V2 (ContractID bigint, StartDT date, EndDT date, DayCount int, WeekDays varchar(255) COLLATE SQL_Latin1_General_CP1_CI_AS);

SET NOCOUNT ON;

DECLARE @VarOldDateFirst int = @@DATEFIRST;
SET DATEFIRST 7;

DECLARE @iFS int;
DECLARE @VarCursor CURSOR;
SET @VarCursor = CURSOR FAST_FORWARD
FOR
    SELECT
        ContractID
        ,dt
        ,dowChar
        ,dowInt
    FROM #Src AS S
    ;

OPEN @VarCursor;

DECLARE @CurrContractID bigint = 0;
DECLARE @Currdt date;
DECLARE @CurrdowChar char(3);
DECLARE @CurrdowInt int;


DECLARE @VarCreateNewInterval bit = 0;
DECLARE @VarTempDT date;
DECLARE @VarTempdowInt int;

DECLARE @LastContractID bigint = 0;
DECLARE @LastStartDT date;
DECLARE @LastEndDT date;
DECLARE @LastDayCount int = 0;
DECLARE @LastWeekDays varchar(255);
DECLARE @LastMonCount int;
DECLARE @LastTueCount int;
DECLARE @LastWedCount int;
DECLARE @LastThuCount int;
DECLARE @LastFriCount int;
DECLARE @LastSatCount int;
DECLARE @LastSunCount int;


FETCH NEXT FROM @VarCursor INTO @CurrContractID, @Currdt, @CurrdowChar, @CurrdowInt;
SET @iFS = @@FETCH_STATUS;
IF @iFS = 0
BEGIN
    SET @LastContractID = @CurrContractID;
    SET @LastStartDT = @Currdt;
    SET @LastEndDT = @Currdt;
    SET @LastDayCount = 1;
    SET @LastMonCount = 0;
    SET @LastTueCount = 0;
    SET @LastWedCount = 0;
    SET @LastThuCount = 0;
    SET @LastFriCount = 0;
    SET @LastSatCount = 0;
    SET @LastSunCount = 0;
    IF @CurrdowInt = 1 SET @LastSunCount = @LastSunCount + 1;
    IF @CurrdowInt = 2 SET @LastMonCount = @LastMonCount + 1;
    IF @CurrdowInt = 3 SET @LastTueCount = @LastTueCount + 1;
    IF @CurrdowInt = 4 SET @LastWedCount = @LastWedCount + 1;
    IF @CurrdowInt = 5 SET @LastThuCount = @LastThuCount + 1;
    IF @CurrdowInt = 6 SET @LastFriCount = @LastFriCount + 1;
    IF @CurrdowInt = 7 SET @LastSatCount = @LastSatCount + 1;
END;

WHILE @iFS = 0
BEGIN

    SET @VarCreateNewInterval = 0;

    -- Contract changes -> start new interval
    IF @LastContractID <> @CurrContractID
    BEGIN
        SET @VarCreateNewInterval = 1;
    END;

    IF @VarCreateNewInterval = 0
    BEGIN
        -- check days of week
        -- are we still within the first week of the interval?
        IF DATEDIFF(day, @LastStartDT, @Currdt) > 6
        BEGIN
            -- we are beyond the first week, check day of the week
            -- have we seen @CurrdowInt before?
            -- we should start a new interval if this is the new day of the week that didn't exist in the first week
            IF @CurrdowInt = 1 AND @LastSunCount = 0 SET @VarCreateNewInterval = 1;
            IF @CurrdowInt = 2 AND @LastMonCount = 0 SET @VarCreateNewInterval = 1;
            IF @CurrdowInt = 3 AND @LastTueCount = 0 SET @VarCreateNewInterval = 1;
            IF @CurrdowInt = 4 AND @LastWedCount = 0 SET @VarCreateNewInterval = 1;
            IF @CurrdowInt = 5 AND @LastThuCount = 0 SET @VarCreateNewInterval = 1;
            IF @CurrdowInt = 6 AND @LastFriCount = 0 SET @VarCreateNewInterval = 1;
            IF @CurrdowInt = 7 AND @LastSatCount = 0 SET @VarCreateNewInterval = 1;

            IF @VarCreateNewInterval = 0
            BEGIN
                -- check the gap between current day and last day of the interval
                -- if the gap between current day and last day of the interval
                -- contains a day of the week that was included in the interval before,
                -- we should create new interval
                SET @VarTempDT = DATEADD(day, 1, @LastEndDT);
                WHILE @VarTempDT < @Currdt
                BEGIN
                    SET @VarTempdowInt = DATEPART(WEEKDAY, @VarTempDT);

                    IF @VarTempdowInt = 1 AND @LastSunCount > 0 BEGIN SET @VarCreateNewInterval = 1; BREAK; END;
                    IF @VarTempdowInt = 2 AND @LastMonCount > 0 BEGIN SET @VarCreateNewInterval = 1; BREAK; END;
                    IF @VarTempdowInt = 3 AND @LastTueCount > 0 BEGIN SET @VarCreateNewInterval = 1; BREAK; END;
                    IF @VarTempdowInt = 4 AND @LastWedCount > 0 BEGIN SET @VarCreateNewInterval = 1; BREAK; END;
                    IF @VarTempdowInt = 5 AND @LastThuCount > 0 BEGIN SET @VarCreateNewInterval = 1; BREAK; END;
                    IF @VarTempdowInt = 6 AND @LastFriCount > 0 BEGIN SET @VarCreateNewInterval = 1; BREAK; END;
                    IF @VarTempdowInt = 7 AND @LastSatCount > 0 BEGIN SET @VarCreateNewInterval = 1; BREAK; END;

                    SET @VarTempDT = DATEADD(day, 1, @VarTempDT);
                END;
            END;
        END;
        -- else
        -- we are still within the first week, so we can add this day to the interval
    END;

    IF @VarCreateNewInterval = 1
    BEGIN
        -- save the new interval into the final table
        SET @LastWeekDays = '';
        IF @LastSunCount > 0 SET @LastWeekDays = @LastWeekDays + 'Sun,';
        IF @LastMonCount > 0 SET @LastWeekDays = @LastWeekDays + 'Mon,';
        IF @LastTueCount > 0 SET @LastWeekDays = @LastWeekDays + 'Tue,';
        IF @LastWedCount > 0 SET @LastWeekDays = @LastWeekDays + 'Wed,';
        IF @LastThuCount > 0 SET @LastWeekDays = @LastWeekDays + 'Thu,';
        IF @LastFriCount > 0 SET @LastWeekDays = @LastWeekDays + 'Fri,';
        IF @LastSatCount > 0 SET @LastWeekDays = @LastWeekDays + 'Sat,';

        INSERT INTO #Dst_V2 
            (ContractID
            ,StartDT
            ,EndDT
            ,DayCount
            ,WeekDays)
        VALUES
            (@LastContractID
            ,@LastStartDT
            ,@LastEndDT
            ,@LastDayCount
            ,@LastWeekDays);

        -- init the new interval
        SET @LastContractID = @CurrContractID;
        SET @LastStartDT = @Currdt;
        SET @LastEndDT = @Currdt;
        SET @LastDayCount = 1;
        SET @LastMonCount = 0;
        SET @LastTueCount = 0;
        SET @LastWedCount = 0;
        SET @LastThuCount = 0;
        SET @LastFriCount = 0;
        SET @LastSatCount = 0;
        SET @LastSunCount = 0;
        IF @CurrdowInt = 1 SET @LastSunCount = @LastSunCount + 1;
        IF @CurrdowInt = 2 SET @LastMonCount = @LastMonCount + 1;
        IF @CurrdowInt = 3 SET @LastTueCount = @LastTueCount + 1;
        IF @CurrdowInt = 4 SET @LastWedCount = @LastWedCount + 1;
        IF @CurrdowInt = 5 SET @LastThuCount = @LastThuCount + 1;
        IF @CurrdowInt = 6 SET @LastFriCount = @LastFriCount + 1;
        IF @CurrdowInt = 7 SET @LastSatCount = @LastSatCount + 1;

    END ELSE BEGIN

        -- update last interval
        SET @LastEndDT = @Currdt;
        SET @LastDayCount = @LastDayCount + 1;
        IF @CurrdowInt = 1 SET @LastSunCount = @LastSunCount + 1;
        IF @CurrdowInt = 2 SET @LastMonCount = @LastMonCount + 1;
        IF @CurrdowInt = 3 SET @LastTueCount = @LastTueCount + 1;
        IF @CurrdowInt = 4 SET @LastWedCount = @LastWedCount + 1;
        IF @CurrdowInt = 5 SET @LastThuCount = @LastThuCount + 1;
        IF @CurrdowInt = 6 SET @LastFriCount = @LastFriCount + 1;
        IF @CurrdowInt = 7 SET @LastSatCount = @LastSatCount + 1;
    END;


    FETCH NEXT FROM @VarCursor INTO @CurrContractID, @Currdt, @CurrdowChar, @CurrdowInt;
    SET @iFS = @@FETCH_STATUS;
END;

-- save the last interval into the final table
IF @LastDayCount > 0
BEGIN
    SET @LastWeekDays = '';
    IF @LastSunCount > 0 SET @LastWeekDays = @LastWeekDays + 'Sun,';
    IF @LastMonCount > 0 SET @LastWeekDays = @LastWeekDays + 'Mon,';
    IF @LastTueCount > 0 SET @LastWeekDays = @LastWeekDays + 'Tue,';
    IF @LastWedCount > 0 SET @LastWeekDays = @LastWeekDays + 'Wed,';
    IF @LastThuCount > 0 SET @LastWeekDays = @LastWeekDays + 'Thu,';
    IF @LastFriCount > 0 SET @LastWeekDays = @LastWeekDays + 'Fri,';
    IF @LastSatCount > 0 SET @LastWeekDays = @LastWeekDays + 'Sat,';

    INSERT INTO #Dst_V2
        (ContractID
        ,StartDT
        ,EndDT
        ,DayCount
        ,WeekDays)
    VALUES
        (@LastContractID
        ,@LastStartDT
        ,@LastEndDT
        ,@LastDayCount
        ,@LastWeekDays);
END;

CLOSE @VarCursor;
DEALLOCATE @VarCursor;

SET DATEFIRST @VarOldDateFirst;

DROP TABLE #Dst_V2;

Answer 6

I was a little bit surprised that the Vladimir's cursor solution was so slow, so I also tried to optimize that version. I did confirm that using a cursor was very slow for me as well.

However, at the cost of using undocumented functionality in SQL Server by appending to a variable while processing a rowset, I was able to create a simplified version of this logic that yields the optimal result and executes much faster than both the cursor and my original solution. So use at your own risk, but I'll present the solution in case it's of interest. It would also be possible to update the solution to use a WHILE loop from one to the maximum row number, seeking to the next row number at each iteration of the loop. This would stick to fully documented and reliable functionality, but would violate the (somewhat artificial) stated constraint of the problem that WHILE loops are not allowed.

Note that if using SQL 2014 was allowed, it is likely that a natively-compiled stored procedure that loops over the row numbers and access each row number in a memory-optimized table would be an implementation of this same logic that would run more quickly.

Here is the full solution, including expanding the trial data set out to about a half million rows. The new solution completes in about 3 seconds and in my opinion is a lot more concise and readable than the previous solution I offered. I'll break out the three steps involved here:

Step 1: pre-processing

We first add a row number to the data set, in the order we will process the data. While doing so, we also convert each dowInt into a power of 2 so that we can use a bitmap to represent which days have been observed in any given grouping:

IF OBJECT_ID('tempdb..#srcWithRn') IS NOT NULL
    DROP TABLE #srcWithRn
GO
SELECT rn = IDENTITY(INT, 1, 1), ContractId, dt, dowInt,
    POWER(2, dowInt) AS dowPower, dowChar
INTO #srcWithRn
FROM #src
ORDER BY ContractId, dt
GO
ALTER TABLE #srcWithRn
ADD PRIMARY KEY (rn)
GO

Step 2: Looping through the contract days in order to identify new groupings

We next loop over the data, in order by row number. We compute only the list of row numbers that form the boundary of a new grouping, then output those row numbers into a table:

DECLARE @ContractId INT, @RnList VARCHAR(MAX), @NewGrouping BIT = 0, @DowBitmap INT = 0, @startDt DATE
SELECT TOP 1 @ContractId = ContractId, @startDt = dt, @RnList = ',' + CONVERT(VARCHAR(MAX), rn), @DowBitmap = DowPower
FROM #srcWithRn
WHERE rn = 1

SELECT 
    -- New grouping if new contract, or if we're observing a new day that we did
    -- not observe within the first 7 days of the grouping
    @NewGrouping = CASE
        WHEN ContractId <> @ContractId THEN 1
        WHEN DATEDIFF(DAY, @startDt, dt) > 6
            AND @DowBitmap & dowPower <> dowPower THEN 1
        ELSE 0
        END,
    @ContractId = ContractId,
    -- If this is a newly observed day in an existing grouping, add it to the bitmap
    @DowBitmap = CASE WHEN @NewGrouping = 0 THEN @DowBitmap | DowPower ELSE DowPower END,
    -- If this is a new grouping, reset the start date of the grouping
    @startDt = CASE WHEN @NewGrouping = 0 THEN @startDt ELSE dt END,
    -- If this is a new grouping, add this rn to the list of row numbers that delineate the boundary of a new grouping
    @RnList = CASE WHEN @NewGrouping = 0 THEN @RnList ELSE @RnList + ',' + CONVERT(VARCHAR(MAX), rn) END 
FROM #srcWithRn
WHERE rn >= 2
ORDER BY rn
OPTION (MAXDOP 1)

-- Split the list of grouping boundaries into a table
IF OBJECT_ID('tempdb..#newGroupingRns') IS NOT NULL
    DROP TABLE #newGroupingRns
SELECT splitListId AS rn
INTO #newGroupingRns
FROM dbo.f_delimitedIntListSplitter(SUBSTRING(@RnList, 2, 1000000000), DEFAULT)
GO
ALTER TABLE #newGroupingRns
ADD PRIMARY KEY (rn)
GO

Step 3: Computing final results based on the row numbers of each grouping boundary

We then compute the final groupings by using the boundaries identified in the loop above to aggregate all dates that fall into each grouping:

IF OBJECT_ID('tempdb..#finalGroupings') IS NOT NULL
    DROP TABLE #finalGroupings
GO
SELECT MIN(s.ContractId) AS ContractId,
    MIN(dt) AS StartDT,
    MAX(dt) AS EndDT,
    COUNT(*) AS DayCount,
    CASE WHEN MAX(CASE WHEN dowChar = 'Sun' THEN 1 ELSE 0 END) = 1 THEN 'Sun,' ELSE '' END + 
    CASE WHEN MAX(CASE WHEN dowChar = 'Mon' THEN 1 ELSE 0 END) = 1 THEN 'Mon,' ELSE '' END + 
    CASE WHEN MAX(CASE WHEN dowChar = 'Tue' THEN 1 ELSE 0 END) = 1 THEN 'Tue,' ELSE '' END + 
    CASE WHEN MAX(CASE WHEN dowChar = 'Wed' THEN 1 ELSE 0 END) = 1 THEN 'Wed,' ELSE '' END + 
    CASE WHEN MAX(CASE WHEN dowChar = 'Thu' THEN 1 ELSE 0 END) = 1 THEN 'Thu,' ELSE '' END + 
    CASE WHEN MAX(CASE WHEN dowChar = 'Fri' THEN 1 ELSE 0 END) = 1 THEN 'Fri,' ELSE '' END + 
    CASE WHEN MAX(CASE WHEN dowChar = 'Sat' THEN 1 ELSE 0 END) = 1 THEN 'Sat,' ELSE '' END AS WeekDays
INTO #finalGroupings
FROM #srcWithRn s
CROSS APPLY (
    -- For any row, its grouping is the largest boundary row number that occurs at or before this row
    SELECT TOP 1 rn AS groupingRn
    FROM #newGroupingRns grp
    WHERE grp.rn <= s.rn
    ORDER BY grp.rn DESC
) g
GROUP BY g.groupingRn
ORDER BY g.groupingRn
GO

Answer 7

Discussion will follow the code.

declare @Helper table(
    rn tinyint,
    dowInt tinyint,
    dowChar char(3));
insert @Helper
values  ( 1,1,'Sun'),
        ( 2,2,'Mon'),
        ( 3,3,'Tue'),
        ( 4,4,'Wed'),
        ( 5,5,'Thu'),
        ( 6,6,'Fri'),
        ( 7,7,'Sat'),
        ( 8,1,'Sun'),
        ( 9,2,'Mon'),
        (10,3,'Tue'),
        (11,4,'Wed'),
        (12,5,'Thu'),
        (13,6,'Fri'),
        (14,7,'Sat');



with MissingDays as
(
    select
        h1.rn as rn1,
        h1.dowChar as StartDay,
        h2.rn as rn2,
        h2.dowInt as FollowingDayInt,
        h2.dowChar as FollowingDayChar
    from @Helper as h1
    inner join @Helper as h2
        on h2.rn > h1.rn
    where h1.rn < 8
    and h2.rn < h1.rn + 8
)
,Numbered as
(
    select
        a.*,
        ROW_NUMBER() over (partition by a.ContractID order by a.dt) as rn
    from #Src as a
)
,Incremented as
(
    select
        b.*,
        convert(varchar(max), b.dowChar)+',' as WeekDays,
        b.dt as IntervalStart
    from Numbered as b
    where b.rn = 1

    union all

    select
        c.*,
        case
            when
                (DATEDIFF(day, d.IntervalStart, c.dt) > 6)      -- interval goes beyond 7 days
            and (
                    (d.WeekDays not like '%'+c.dowChar+'%')     -- the new week day has not been seen before
                or 
                    (DATEDIFF(day, d.dt, c.dt) > 7)
                or 
                    (
                        (DATEDIFF(day, d.dt, c.dt) > 1)
                        and
                        (
                        exists( select
                                    e.FollowingDayChar
                                from MissingDays as e
                                where e.StartDay = d.dowChar
                                and rn2 < (select f.rn2 from MissingDays as f
                                            where f.StartDay = d.dowChar
                                            and f.FollowingDayInt = c.dowInt)
                                and d.WeekDays like '%'+e.FollowingDayChar+'%'
                            )
                        )
                    )
                )
            then convert(varchar(max),c.dowChar)+','
            else
                case
                    when d.WeekDays like '%'+c.dowChar+'%'
                    then d.WeekDays
                    else d.WeekDays+convert(varchar(max),c.dowChar)+','
                end
        end,
        case
            when
                (DATEDIFF(day, d.IntervalStart, c.dt) > 6)      -- interval goes beyond 7 days
            and (
                    (d.WeekDays not like '%'+c.dowChar+'%')     -- the new week day has not been seen before
                or
                    (DATEDIFF(day, d.dt, c.dt) > 7)             -- there is a one week gap
                or 
                    (
                        (DATEDIFF(day, d.dt, c.dt) > 1)         -- there is a gap..
                        and
                        (
                        exists( select                          -- .. and the omitted days are in the preceeding interval
                                    e.FollowingDayChar
                                from MissingDays as e
                                where e.StartDay = d.dowChar
                                and rn2 < (select f.rn2 from MissingDays as f
                                            where f.StartDay = d.dowChar
                                            and f.FollowingDayInt = c.dowInt)
                                and d.WeekDays like '%'+e.FollowingDayChar+'%'
                            )
                        )
                    )
                )
            then c.dt
            else d.IntervalStart
        end
    from Numbered as c
    inner join Incremented as d
    on d.ContractID = c.ContractID
    and d.rn = c.rn - 1
)
select
    g.ContractID,
    g.IntervalStart as StartDT,
    MAX(g.dt) as EndDT,
    COUNT(*) as DayCount,
    MAX(g.WeekDays) as WeekDays
from Incremented as g
group by
    g.ContractID,
    g.IntervalStart
order by
    ContractID,
    StartDT;

@Helper is to cope with this rule:

If the gap between current day and last day of the interval contains a day of the week that was included in the interval before, we should create new interval

It allows me to list day names, in day number order, between any two given days. This is used when deciding if a new interval should start. I populate it with two week's worth of values to make wrapping around a weekend easier to code.

There are cleaner ways to implement this. A full "dates" table would be one. There's probably a clever way with day number and modulo arithmetic, too.

The CTE MissingDays is to generate a list of day names between any two given days. It is handled in this clunky way because the recursive CTE (following) does not allow aggregates, TOP(), or other operators. This is inelegant, but it works.

CTE Numbered is to enforce a known, gap-free sequence on the data. It avoids a lot of comparisons later.

CTE Incremented is where the action happens. In essence I use a recursive CTE to step through the data and enforce the rules. The row number generated in Numbered (above) is used to drive the recursive processing.

The seed of the recursive CTE simply gets the first date for each ContractID and initialises values that will be used to decide if a new interval is required.

Deciding if a new interval should start requires the current interval's start date, day list and the length of any gap in the calendar dates. These may be reset or carried forward, depending on the decision. Hence the recursive part is verbose and a little repetitive, as we have to decide whether to start a new interval for more than one column value.

The decision logic for columns WeekDays and IntervalStart should have the same decision logic - it can be cut-and-pasted between them. If the logic for starting a new interval were to change this is the code to alter. Ideally it would be abstracted, therefore; doing this in a recursive CTE may be challenging.

The EXISTS() clause is the outfall of not being able to use aggregate functions in a recursive CTE. All it does is see if the days falling within a gap are in the current interval already.

There is nothing magic about the nesting of the logic clauses. If it is clearer in another conformation, or using nested CASEs, say, there is no reason to keep it this way.

The final SELECT is to give the output in the format desired.

Having the PK on Src.ID is not useful for this method. A clustered index on (ContractID,dt) would be nice, I think.

There are a few rough edges. The days are not returned in dow sequence, but in the calendar sequence they appear in the source data. Everything to do with @Helper is klunky and could be smoothed. I like the idea of using one bit per day and using binary functions instead of LIKE. Separating some of the auxilliary CTEs into temp table with proper indexes would undoubtedly help.

One of the challenges with this is that a "week" does not align with a standard calendar, but rather is driven by the data, and resets when it is determined that a new interval should begin. A "week", or at least an interval, can be from one day long to spanning the entire dataset.

---

For interests sake, here's the estimated costs against Geoff's sample data (thanks for that!) after various changes:

                                             estimated cost

My submission as is w/ CTEs, Geoff's data:      791682
Geoff's data, cluster key on (ContractID, dt):   21156.2
Real table for MissingDays:                      21156.2
Numbered as table UCI=(ContractID, rn):             16.6115    26s elapsed.
                  UCI=(rn, ContractID):             41.9845    26s elapsed.
MissingDays as refactored to simple lookup          16.6477    22s elapsed.
Weekdays as varchar(30)                             13.4013    30s elapsed.

The estimated and actual number of rows differ wildly.

The plan has a table spoo, likely as a result of the recursive CTE. Most of the action is in a worktable coming off that:

Table 'Worktable'.   Scan count       2, logical reads 4 196 269, physical reads 0, read-ahead reads 0, lob logical reads 0, lob physical reads 0, lob read-ahead reads 0.
Table 'MissingDays'. Scan count 464 116, logical reads   928 232, physical reads 0, read-ahead reads 0, lob logical reads 0, lob physical reads 0, lob read-ahead reads 0.
Table 'Numbered'.    Scan count 484 122, logical reads 1 475 467, physical reads 0, read-ahead reads 0, lob logical reads 0, lob physical reads 0, lob read-ahead reads 0.

Just the way recursive's implemented, I guess!