Exploring Scalar Solutions to Complex Data Math

There are many functions and tools available to database professionals that can solve data math challenges, regardless of complexity. A recent problem came across my desk that called for not only a valid solution, but one that performs as optimally as possible when included in a scalar User-Defined-Function (UDF).

This left me asking: How often these problems impact us and what general solutions can we find that may solve them more easily for us?

The impetus for this was a simple question, “How many instances of days occur between two dates?”. For example: “How many Mondays were there between 1/1/2024 and 6/17/2024?” Or: “How many Tuesdays and Thursdays are there in total from 9/2/2022 and 9/1/2025?”

This sounds super-simple at first, but as I dove in for a universal solution, I found myself coming back to either iteration, calendar tables, or exceptionally long SQL that replaced iteration with copies of the same code over-and-over. There are no built-in functions in SQL Server that do what I wanted.

Notes on DATEFIRST and SQL Server Defaults

SQL Server by default assigns a number to each day of the week that can be used in date math and identification. The defaults are as follows:

This means that the default in SQL Server is for Sunday to be the first day of the week. Any date math that identifies the day of the week will return “1” for the day corresponding to Sunday, like this:

SELECT DATEPART(DW, '12/31/2023') AS DayOfWeekResult;

12/31/2023 was a Sunday, so the results are as follows:

Most developers will go along with this convention as it is the default and easy enough to accommodate with consistent code. The default can be changed at the connection level, using SET DATEFIRST:

SET DATEFIRST 1;

For my session, the first day of the week is now Monday, and the day of the week for 12/31/2023 is now going to be a different value:

Sunday is defined as the last day of the week and therefore is indicated with a day-of-week of 7. Keep in mind that this setting only applies to a given session and all other processes in SQL Server will continue to return results as if SET DATEFIRST were never invoked.

Maintaining the defaults is far easier than changing the first day of the week in SQL Server. Regardless of defaults, this article will maintain Sunday as the first day of the week and not change this as part of the problem-solving done here. Before continuing, let’s return this setting to the default, to prevent any potential confusion later on:

SET DATEFIRST 7;

Note that the code in this article assumes that this default is enabled. If your SQL Server has another default, or if defaults are adjusted on a session-by-session basis, then the days of the week used here will also adjust accordingly.

Solutions That Do Not Perform Well Enough

In this section I will cover some of the solutions that work, often seemly fast enough, but for dealing with lot of rows, they start slowing down the output to a level that is not desirable.

Simple Iteration

Iteration is the easiest solution to the problem. Consider the first example question: “How many Mondays were there between 1/1/2024 and 6/17/2024?” We can set a variable equal to the start date and iterate through every date between it and the end date, counting instances of Mondays along the way, like this:

DECLARE @StartDate DATE = '1/1/2024';
DECLARE @EndDate DATE = '6/17/2024';
DECLARE @CurrentDate DATE = @StartDate;
DECLARE @Result INT = 0;
WHILE @CurrentDate <= @EndDate
BEGIN
	IF DATEPART(DW, @CurrentDate) = 2
	BEGIN
		SELECT @Result = @Result + 1;
	END
	SELECT @CurrentDate = DATEADD(DAY, 1, @CurrentDate);
END
SELECT @Result;

Nothing fancy here – we get the result by iterating 168 times and done. The wider the date range, the more effort is needed to solve for the result. If joined into a table and directly applied to a column, then the effort would increase based both on row counts and date ranges.

Of course, when you execute this directly, any slowness is imperceptible. But when you try to use this function with large datasets, the iterations become more and more noticeable. In this article I won’t delve into the testing of the functions, but it is important to consider the amount of data your code may be used with and test with as much more than that that you can.

Tactical Iteration

An alternative that is less painful, but also quite methodical is to iterate by weeks prior to iterating by days. We know that for each 7-day span, each day of the week will occur once.

Therefore, iteration can be accomplished via incrementing by weeks, until within seven days of the @EndDate. This solution can be implemented using the following T-SQL:

DECLARE @StartDate DATE = '1/1/2024';
DECLARE @EndDate DATE = '6/17/2024';
DECLARE @CurrentDate DATE = @StartDate;
DECLARE @Result INT = 0;
--get the number of full weeks past
WHILE DATEADD(DAY, 7, @CurrentDate) <= @EndDate
BEGIN
    SELECT @Result = @Result + 1;
    SELECT @CurrentDate = DATEADD(DAY, 7, @CurrentDate);
END
--then see if there is a Monday in the remainder of the week.
WHILE @CurrentDate <= @EndDate
BEGIN
    IF DATEPART(DW, @CurrentDate) = 2
    BEGIN
        SELECT @Result = @Result + 1;
    END
    SELECT @CurrentDate = DATEADD(DAY, 1, @CurrentDate);
END
SELECT @Result;

This solution has two WHILE blocks. The first one loops by increments of seven days, adding one to the result each tie. The second WHILE loop performs the same logic as the previous solution, iterating the remainder of days until reaching @EndDate. While this iterates less, it’s more complex and still relies on WHILE loops that have to iterate some number of times each to return a result.

It is useful to be reminded that any solution built here is likely to be encapsulated into a scalar function. Once converted to a function, it can be embedded into any part of a query, including aggregation, filtering, and ordering. Therefore, any performance challenges identified, even if minor, will be magnified when applied to any part of a query against data of potentially any size!

The following code is the natural extension of what was done here:

CREATE FUNCTION dbo.ReturnDaysBetweenDates_Iteration
    (@StartDate DATE, @EndDate DATE)
RETURNS INT
AS
BEGIN
    DECLARE @CurrentDate DATE = @StartDate;
    DECLARE @Result INT = 0;
    WHILE @CurrentDate <= @EndDate
    BEGIN
        IF DATEPART(DW, @CurrentDate) = 2
        BEGIN
            SELECT @Result = @Result + 1;
        END
        SELECT @CurrentDate = DATEADD(DAY, 1, @CurrentDate);
    END
    RETURN @Result;
END

Once wrapped into a function, it may be reused anywhere, including inside additional functions, stored procedures, or views.

While reusing optimal object can result in great savings, reusing suboptimal code can causes inefficiencies being multiplied exponentially, which for the sake of performance can be absolutely crushing!

For this reason, maximizing efficiency is quite important here, so it is important to test, and in some cases, looking for even more efficient, if not as straightforward solutions that will work fast enough when used in any situation! In this case, needing to iterate was deemed to be a suboptimal solution, so I continued at it looking for a better solution.

Solving it All-at-Once

The key to solving this date problem lies in the fact that the effort needed to compute the answer does not need to be linear to the time frame analyzed.

When solving this sort of issues, it can be a good idea to look at the problem from a bit higher level. While you should always be careful to not extend the scope too far, sometimes a general solution can also be the better solution.

In this case, it was determined that it would also be ideal to expand the solution to allow for calculating day-of-the-week counts for any individual day or combination of days. Weekdays are a common data request, but different fields will require other combinations of days based on differing business models. For the demonstration here, the example of solving for Monday will be completed first, and then a more complex scenario involving multiple days.

Calculating days of the week over a one-hundred-year span should not require more effort than over a one-month span. A visualization can help in seeing a better solution:

In this example, the start date for the date range is on a Tuesday and the @EndDate is on a Friday, 2 weeks later. To calculate the number of Mondays in this time frame requires 2 bits of math:

This was accomplished earlier via iteration, but by visualizing the problem, it becomes clear that there are 2 calculations that need to be completed. The first is trivial: take the number of days in the date range and divide by seven. The result provides the first part of the solution:

The remaining challenge is to count the extra days at the end of the date range, which will be a number from 0-6. Visually, it is easy to see how many days remain and which days of the week they are, but how can this programmatically be calculated? The number of days and which days of the week are represented are both required to solve this problem, without any human/visual inspection.

The first and simplest scenario is one where the number of days in the date range is an even multiple of seven:

In this scenario, the number of Mondays will always be the count of days in the date range divided by seven:

SELECT (DATEDIFF(DAY, @StartDate, @EndDate) + 1) / 7;

This can also be simplified to:

SELECT DATEDIFF(WEEK, @StartDate, @EndDate) / 7;

The other two scenarios are:

For each of these, the count of full weeks is computed as it was above and 1 is added to it if the remaining days happen to include a Monday.

Scenario #1 looks like this:

If the start day (2) is less than or equal to the end day (5), which it is, then whether a Monday exists in that range depends solely on if the start day is less than or equal to 1 or not. Since it is, we can add one and the total number of Mondays is 3, and given by the following T-SQL:

SELECT @MondayCount = DATEDIFF(WEEK, @StartDate, @EndDate) / 7 +
CASE WHEN @DayOfWeekStart > 1 THEN 0 ELSE 1 END;

Alternatively, if the start day (2) is greater than the end day (5), then the extra Monday would not have been added on, and the count would have been two, rather than three.

There are two more problems to solve here:

The first is an extension of the solution tackled above. Instead of assuming that Monday is the day to check for, that day becomes a variable that we can call @DayToCheck. For example, if the @DayToCheck was Saturday, then the visualization would look like this:

With the Monday assumption gone, the resulting math becomes this:

The one mathematical issue to solve first is the fact that the start date and end date can be before or after each other. The simplest way to handle this situation is to iterate through each additional day…but…we are trying our best to avoid iteration here, so that option will be discarded.

To solve this problem in a way that can also be applied to a scenario where we want to check any number of days-of-the-week, we will consider a bitmap-based solution. Imagine that there is a 7-bit bitmap that indicates the days we are searching for:

The days are backwards so that the values will appear forwards. In other words, the first bit (2⁰) is Sunday, the second bit (2¹) is Monday, and so on. If the goal is to validate how many Saturdays are in a date span, then the bitmap representation for this input would be:

The binary number 1000000 is equal to decimal 64.

Why use a bitmap? Isn’t that complex and hard to understand? It certainly isn’t as simple as the iteration code was, but the next step will help make this a bit clearer.

Consider how the day count is calculated: We start with the number of full 7-day weeks in the date span and then add to it depending on what the remainder days contain. Therefore, we can also create a bitmap for the remainder that indicates the days within it.

Consider the visual representation of the problem that we are tackling here:

There are 2 complete weeks of 7 days, starting from @StartDate, which is a Monday. There are then four leftover days. These remainder days can be summarized in a bitmap as well:

The resulting number in binary is 11110, or 30 in decimal.

With these two bitmaps, the real magic trick can be performed: Overlap the two binary numbers and perform a bitwise AND against them:

By comparing the days-to-be-checked with the remainder, any overlap can be calculated using the single operation above. In this scenario, the lack of overlap confirms that the date range provided does not contain any extra Saturdays within the remainder days. As a huge bonus, this process works for any combination of days-to-check and any combination of remainder days.

Creating the Function in T-SQL

With the binary math out of the way, a solution can be built in T-SQL that accomplishes this task for all possible inputs. The following function accepts a start date, end date, and a bit for each day of the week and will return the total number of times the days-to-check appear within the date range:

CREATE FUNCTION dbo.DaysWithinDateRangeCount
(   @StartDate DATE,
    @EndDate DATE,
    --pick as many days of the week as you wish to count
    @Monday BIT,
    @Tuesday BIT,
    @Wednesday BIT,
    @Thursday BIT,
    @Friday BIT,
    @Saturday BIT,
    @Sunday BIT)
RETURNS INTEGER
AS
BEGIN
       --set up the bitmap for the days in the parameters.
    DECLARE @DayBitmap INT = CAST(@Saturday AS INT) * 64 +
                                CAST(@Friday AS INT) * 32 + 
                                CAST(@Thursday AS INT) * 16 +
                                CAST(@Wednesday AS INT) * 8 +
                                CAST(@Tuesday AS INT) * 4 + 
                                CAST(@Monday AS INT) * 2 + 
                                CAST(@Sunday AS INT) * 1;

    -- Total days in the search range.
    DECLARE @DayCountTotal INT = DATEDIFF(DAY, @startDate, @endDate) + 1; 

    -- Total number of complete weeks in the search range.
    DECLARE @WeekCountFull INT = @DayCountTotal / 7; 

    -- Additional days leftover in the search range.
    DECLARE @Remainder INT = @DayCountTotal % 7; 

    -- Number of total days to check within the search range.
    DECLARE @DaysToCheckCount INT = BIT_COUNT(@DayBitmap); 

    -- The day of week for the last day in the search range.
    DECLARE @LastDay INT = DATEPART(WEEKDAY, @endDate); 

    DECLARE @RemainderBitMap INT =
        CASE WHEN @remainder = 0 THEN 0
         WHEN @remainder = 1 THEN POWER(2, @LastDay - 1)
         WHEN @remainder = 2 THEN POWER(2, @LastDay - 1) 
                     + POWER(2, IIF(@LastDay = 1, 6, @LastDay - 2))
         WHEN @remainder = 3 THEN POWER(2, @LastDay - 1) 
                     + POWER(2, IIF(@LastDay = 1, 6, @LastDay - 2)) 
                     + POWER(2, IIF(@LastDay - 2 < 1, @LastDay + 4, @LastDay - 3))
         WHEN @remainder = 4 THEN POWER(2, @LastDay - 1) 
                     + POWER(2, IIF(@LastDay = 1, 6, @LastDay - 2)) 
                     + POWER(2, IIF(@LastDay - 2 < 1, @LastDay + 4, @LastDay - 3)) 
                     + POWER(2, IIF(@LastDay - 3 < 1, @LastDay + 3, @LastDay - 4))
         WHEN @remainder = 5 THEN POWER(2, @LastDay - 1) 
                     + POWER(2, IIF(@LastDay = 1, 6, @LastDay - 2)) 
                     + POWER(2, IIF(@LastDay - 2 < 1, @LastDay + 4, @LastDay - 3)) 
                     + POWER(2, IIF(@LastDay - 3 < 1, @LastDay + 3, @LastDay - 4)) 
                     + POWER(2, IIF(@LastDay - 4 < 1, @LastDay + 2, @LastDay - 5))
         WHEN @remainder = 6 THEN POWER(2, @LastDay - 1) 
                     + POWER(2, IIF(@LastDay = 1, 6, @LastDay - 2)) 
                     + POWER(2, IIF(@LastDay - 2 < 1, @LastDay + 4, @LastDay - 3)) 
                     + POWER(2, IIF(@LastDay - 3 < 1, @LastDay + 3, @LastDay - 4)) 
                     + POWER(2, IIF(@LastDay - 4 < 1, @LastDay + 2, @LastDay - 5)) 
                     + POWER(2, IIF(@LastDay - 5 < 1, @LastDay + 1, @LastDay - 6))
               --no remainder = 7, because that is a full week already
    END
    RETURN (@WeekCountFull * @DaysToCheckCount) 
                        + BIT_COUNT(@RemainderBitMap & @DayBitmap);
END;

Note: this function will only work as is in SQL Server 2022 due to the use of the BIT_COUNT function. In this article, there is a function that you can use to replace the BIT_COUNT function if you are using a previous version of SQL Server.

The bitwise math may seem a bit dizzying (and it wasn’t simple to write!), but it is the same basic math repeated over-and-over. The steps taken in the function are as follows:

Whereas @DayBitMap is somewhat easy to calculate, as the days of the week to check for are entered explicitly as input variables to the stored procedure, @RemainderBitMap does require a bit more logic.

The complexity is in taking a number and determining the remainder (base 7) and converting that into an identically formatted day-of-week bitmap that can be compared directly to @DayBitMap. A bitmap is just a sequence of bits, each that represent a different power of 2, all added up.

For example, if the remainder were 3, then there are three leftover days that need to be mapped to their corresponding days-of-the-week:

WHEN @remainder = 3 THEN POWER(2, @LastDay - 1) 
                + POWER(2, IIF(@LastDay = 1, 6, @LastDay - 2)) 
                + POWER(2, IIF(@LastDay - 2 < 1, @LastDay + 4, 
                                                @LastDay - 3))

The embedded IIF conditional is used to check for when the day of week wraps around from 7 back to 1. This ensures that we do not continue counting days of the week 8, 9, 10, etc…and instead wrap around from Saturday (last day of the week) back to Sunday (first day of the week). The POWER function raises each subsequent power of 2 based on the day of week, adding the results to generate the desired bitmap.

The function can easily be tested for the example above:

SELECT dbo.DaysWithinDateRangeCount('12/2/2024', '12/19/2024', 0, 0, 0, 0, 0, 1, 0) 
                                                                          AS Result;

The result comes back with the expected value:

Note too that executing the following:

SELECT dbo.DaysWithinDateRangeCount('1/1/2024', '6/17/2024', 1, 0, 0, 0, 0, 0, 0) 
                                                                        AS Result;

Will return the same 25 instances as the original example.

Consider the same date range, but we would like to check for Monday, Wednesday, and Friday. This time, the days to check are as follows:

The remainder can be expressed as this binary value (same as before):

The bitwise AND of these numbers is 0101010 & 0011110 = 0001010

The result for days between each date is:

The function can be tested to validate this answer:

SELECT dbo.DaysWithinDateRangeCount('12/2/2024', '12/19/2024', 1, 0, 1, 0, 1, 0, 0) 
                                                                         AS Result;

The result returned is:

From here, an arbitrarily random complex problem can be solved, such as occurrences of Sunday and Wednesday from 1/17/2010 through 12/17/2025:

SELECT dbo.DaysWithinDateRangeCount('1/17/2010', '12/17/2025', 0, 0, 1, 0, 0, 0, 1) 
                                                                          AS Result;

The result that is returned is:

We can (for fun of course!) confirm this result manually:

Without even executing this code, you should immediately be able to see that it will execute far faster than the looping code because it is all simple scalar math operations. No matter the distance from start to finish, and no matter which days you are counting, it will be as fast for very short distances between dates or far faster when the date values are farther apart.

Conclusion

There are many possible ways to solve the problem presented in this article. There are also many other date-math problems that can be solved in similar ways. The goal was to think a bit creatively (and mathematically) and consider other ways to get the correct result with fewer iterative steps.

The beauty of re-solving a problem in this manner is that the computing power to execute this function is similar, regardless of the parameters chosen. Other common solutions to a problem like this are iterative or contain many steps, increasing their cost.

The added fun is in thinking up something new and different. This strategy can most certainly be used elsewhere in coding to solve similar problems where an overlap of data sets needs to be evaluated.

Have you encountered any other problems like this – where creative, unusual, or interesting solutions saved the day?

The post Exploring Scalar Solutions to Complex Data Math appeared first on Simple Talk.