SQL getting proportional numbers

Viewed 63

I have 2 tables in my Microsoft SQL Server database:

customers

╔════╦════════════╦══════════╗
║ id ║ open_date  ║ eligible ║
╠════╬════════════╬══════════╣
║  1 ║ 01/12/2020 ║ True     ║
║  2 ║ 05/12/2020 ║ True     ║
║  3 ║ 02/12/2020 ║ False    ║
╚════╩════════════╩══════════╝

registered_customers

╔════╦════════════╗
║ id ║  reg_date  ║
╠════╬════════════╣
║  1 ║ 04/12/2020 ║
║  2 ║ 07/12/2020 ║
║  3 ║ 22/12/2020 ║
╚════╩════════════╝

How would I go about calculating the following 2 numbers:

  1. For customers who have the eligible status of true, what proportion actually followed up and registered?

  2. What is the median duration between opening an account and registering?

I know how to get the individual counts for one, but not sure how to get proportion. For two would I join on the id, then subtract duration of open date from reg date, then get the median? Thanks

3 Answers
with main_data as (select 
customers.*,
registered_customers.reg_date,



from customers
left join registered_customers
on customers.id = registered_customers.id
),

select 
count (case when main_data.eligible = True
            then 1 else 0 end) as total_users_with_eligibility,

count (case when main_data.eligible = True and
                 main_data.reg_date is not null
            then 1 else 0 end) as total_users_with_eligibility_and_registration,


(count (case when main_data.eligible = True and main_data.reg_date is not null
            then 1 else 0 end)
 /
count (case when main_data.eligible = True
            then 1 else 0 end)) * 100
       as ratio_elegibility_and_registered_over_just_eligibile

from main_data 

Hmmmm . . . calculating medians is tricky. One method uses percentile_cont():

select avg(is_registered), max(median)
from (select c.*,
            (case when rc.id is not null then 1.0 else 0 end) as is_registered,
             percentile_cont(0.5) within group (case when eligible = 'true' then datediff(day, c.open_date, rc.reg_date) end) over () as median
      from customers c left join
           registered_customers rc
           on rc.id = c.id
     ) c

The first problem is easy to solve - it is just a ratio. I set up a script to mimic your data, but I did add one more customer who never registered so I could look at the results. Let's start with making some variables and then finding the ratio of eligible customers who ended up registering. I chose BIT as the data type for eligible because SQL Server doesn't have a Boolean data type and BIT is the most common substitute with 1 meaning TRUE and 0 meaning FALSE.

DECLARE @Customers AS TABLE (
    Id INT NOT NULL, 
    Open_Date DATE NOT NULL, 
    Eligible BIT NOT NULL
);

INSERT INTO @Customers(Id, Open_Date, Eligible) 
VALUES (1,'2020-12-01',1),(2,'2020-12-05',1), 
(3,'2020-12-02',0),(4,'2020-12-07',1);

DECLARE @RegisteredCustomers AS TABLE(
    Id INT NOT NULL, 
    RegDate DATE NOT NULL
);

INSERT INTO @RegisteredCustomers(Id, RegDate)
VALUES (1,'2020-12-04'), (2,'2020-12-07'), 
(3,'2020-12-22');

--for customers who have eligbile status of True, what proportion actually followed up and registered?
;WiTH EligibleCustomers AS (
    SELECT Open_Date FROM @Customers WHERE Eligible = 1
), RegisteredEligibleCustomers AS (
    SELECT c.Open_Date, r.RegDate  
    FROM @Customers AS c
    INNER JOIN @RegisteredCustomers AS r ON r.Id = c.Id 
    WHERE c.Eligible = 1
)
SELECT (SELECT CAST(COUNT(*) AS decimal(5,2)) FROM RegisteredEligibleCustomers) / 
(SELECT CAST(COUNT(*) AS decimal(5,2)) FROM EligibleCustomers) 
AS [ProportionOfEligibleWhoRegistered]; 

Note the CAST function - you must CAST away from the INT that COUNT returns or you will get integer division and the results will not be what you expect! For this data, the result is 0.66666666 which is correct (out of 4 customers, 3 were eligible but only 2 registered, so 2 / 3 is the proportion). If you do not CAST, it will return 0 from the integer division which is not what you want.

As far as the median, that is trickier. I do not know what version of SQL Server you are using, but I will use a common trick that works as far back as SQL Server 2000. Perform two queries - get the MAX value from the first half of the results sorted ascending, then get the MIN value from the first half of the results sorted descending and then just take their average. Yes, there are other functions you can use, but they depend on the version of SQL Server you have. If you want to read about other techniques and their performance, check out the article https://sqlperformance.com/2012/08/t-sql-queries/median

So, for the second question you pose, I will start with the same sample data to try things out.

;WITH RegisteredEligibleCustomers AS (
    SELECT DATEDIFF(day, c.Open_Date, r.RegDate) AS [DaysToRegister]
    FROM @Customers AS c
    INNER JOIN @RegisteredCustomers AS r ON r.Id = c.Id 
    WHERE c.Eligible = 1
)
SELECT 
(
    (SELECT MAX(DaysToRegister) FROM (SELECT TOP 50 PERCENT DaysToRegister 
    FROM RegisteredEligibleCustomers ORDER BY DaysToRegister) AS u) +  
    (SELECT MIN(DaysToRegister) FROM (SELECT TOP 50 PERCENT DaysToRegister 
    FROM RegisteredEligibleCustomers ORDER BY DaysToRegister DESC) AS v)
) / 2.0 AS [MedianDaysToRegister];

Note that you can change the first parameter to the DATEDIFF function if you want the result in a different unit of time - I arbitrarily chose days.

For the sample data, out of the 4 customers, 3 were eligible but only 2 registered. It took 1 customer 3 days to register and another customer 2 days to register, so the median is 2.5 here.

Related