Design of experiment: Is there correlation between performance vs. employee attrition?

Question

Background:

Assume company X has a cyber security incident team. One of their key performance/risk indicators is how fast they triage the security incidents. For the past 2 quarters, COO observed decreased amounts of incidents triaged within ideal/expected triage time.

Task:

Find out why there has been a decreased amounts of incidents triaged within ideal triage time.

Approach:

1. Employees come and go. There has been a continuous resignations. Test if decreased number of team members has correlation with triage time.

Problem:

I'm using simple linear regression. Triage time (in minutes) is response variable (y) and 'number of people joined/left' is explanatory variable.
Triage time data range is from 0~940000 minutes, and number of people is only from -4 to 2. -4 being when team lost 4 people and 2 being where they had 2 extra members from where they began the team.

Below is what it looks like (fake data), and I have 65000+ rows

The result is saying the model isn't significant.

Result:

Regression Statistics
Multiple R 0.004008821
R Square 1.60706E-05
Adjusted R Square 8.26758E-07
Standard Error 642.2262936
Observations 65601
Significance F 0.30453744

I think maybe there is a better way to study relationship between employee attrition versus triage time.

Question is

Is it okay to compare the variables that one has much wider range (y) compare to another (x)? I'm thinking if I was supposed to perform any data transformation so I can somehow emphasize the employee attrition. Do you see anything wrong with the way I design the linear regression?

Answer 1

The short answer is: No, it is not a problem to use predictor variables in linear regression that are on a differently scale than the outcome. An extreme example is the use of dummy variables to represent factors: Dummy variables only contain 0's and 1's, but this is completely fine, as long as the other assumptions of your model hold — and here I can see problems with the scenario described above:

1.) The assumption that staffingPower and triageTime are linearly related may not hold. E.g. a staffing power of -2 to 2 may be fine, but at -3 to -4 the efficiency of the team may break down.

• Solution a): try what happens when encoding staffing_power as categorical variables (i.e. 5 dummy variables).
• Solution b): try what happens if you go non-linear (e.g. by polynomial expansion of your predictor staffing power)

(both of these solutions will also "emphasize" potential effects of employee attrition)

2.) The assumption of independent samples/no autocorrelation may not hold in your case, as the response time to incidents may be temporally correlated. This will not affect your regression coefficients, but may bias the error estimates. You can check for auto-correlation by using an auto-correlation plot, Durbin-Watson-Test, and/or by plotting the residuals of your regression against createDate (and then see whether any kind of pattern emerges other than random noise).

3.) The assumption of homoscedascity may not hold: Maybe your model predictions for long triageTimes are more variable than the predictions for short triageTimes? Check by plotting residuals against predicted values. If there is heteroscedascity you will see a "funnel" like relationship. Potential solution: transforms of triageTime (e.g. log(triageTime)).