1
$\begingroup$

Stats noob here. Is it possible and appropriate to use an independent t test to compare attrition rates of one department to another department? For example, Department A has 15 teams and Department B has 15 teams. I have attrition rates for each of the 30 teams which I calculated based on the employees in those teams. Assuming all of the assumptions of a t test are met, is it appropriate to use a t test to say that the mean difference of one department’s attrition is or isn’t statistically significant from the other. I guess my real question is can you run the same types of statistical tests you can do on individual samples on aggregated data?

If not, what is the correct way to go about comparing aggregated data?

d = {'Dept': ['A', 'A', 'A', 'A', 'A', 'A', 'A', 'A', 'A', 'A', 'A', 'A', 'A', 'A', 'A', 'B', 'B', 'B', 'B', 'B', 'B', 'B', 'B', 'B', 'B', 'B', 'B', 'B', 'B', 'B'], 'Team': [1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15, 16, 17, 18, 19, 20, 21, 22, 23, 24, 25, 26, 27, 28, 29, 30], 'Rate': [0.0953, 0.0899, 0.0888, 0.0999, 0.0771, 0.084, 0.0942, 0.1165, 0.0917, 0.0989, 0.0843, 0.1004, 0.0884, 0.0818, 0.0879, 0.1258, 0.1189, 0.1367, 0.0826, 0.1192, 0.111, 0.1098, 0.0883, 0.0772, 0.139, 0.0966, 0.1233, 0.1199, 0.0809, 0.0997]}
df = pd.DataFrame(data=d)
df
$\endgroup$
5
  • $\begingroup$ Can you show (an excerpt of) your data? $\endgroup$ Jul 10, 2019 at 0:49
  • $\begingroup$ Hi @FransRodenburg I updated my question with some of the data aggregated by team $\endgroup$
    – Insu Q
    Jul 10, 2019 at 2:03
  • $\begingroup$ Can you please post this as a csv or a dataframe? I'm not interested in parsing latex tables $\endgroup$ Jul 10, 2019 at 2:07
  • $\begingroup$ Added data as df. $\endgroup$
    – Insu Q
    Jul 10, 2019 at 2:21
  • $\begingroup$ @InsuQ I've done my own analysis below. I think a t-test is fine in this instance. $\endgroup$ Jul 10, 2019 at 2:46

1 Answer 1

2
$\begingroup$

Yes and no. There are definitely better ways to do this. Let me explain.

People in each team are more similar than people between teams. Teams in departments are are more alike than teams between departments. Accounting for this correlation can help give more robust inferences. If you know the attrition rates for each team (and thus, for each department) it may make more sense to fit a hierarchical model of attrition.

However a t test (or perhaps a test of proportions in the case of attrition rates) can still be useful, despite the things I have mentioned. If the attrition rates are really different then an appropriate test will pick that up.

I should mention that the approach you take will depend on your data. Can we see some of it?

EDIT

OK, we've got data now. This isn't exactly what I was thinking the data would look like, so maybe a hierarchical model isn't really the way to go.

Here is a plot of the data. You can see that the attrition rates for department B are much more variable than A. I've jittered them horizontally but not vertically for ease on viewing. Welch's t-test (which accounts for this difference in variance) should probably be used.

enter image description here

Here is what the t-test shows:

Welch Two Sample t-test

data:  Rate by Dept
t = -2.8981, df = 20.084, p-value = 0.008865
alternative hypothesis: true difference in means is not equal to 0
95 percent confidence interval:
 -0.028636482 -0.004670184
sample estimates:
mean in group A mean in group B 
      0.0919400       0.1085933 

Seems to be a difference between groups, with Department B having the larger attrition rate. Interesting in its own right.

And from there, you could stop. But, because I'm curious, let's compare it to a fancier model.

Because no team has exactly 0 attrition rate, I think we can model the rates as being beta distributed. We'll model the mean and the dispersion as depending on the department. Using brms to do the modelling...

model = brm(bf(Rate~Dept, phi~Dept),
             family = Beta(),
              data = dept)


              Estimate Est.Error l-95% CI u-95% CI Eff.Sample Rhat
Intercept        -2.29      0.03    -2.36    -2.22       4149 1.00
phi_Intercept     6.76      0.41     5.90     7.49       3114 1.00
DeptB             0.19      0.07     0.05     0.32       2423 1.00
phi_DeptB        -1.40      0.56    -2.55    -0.28       2858 1.00

The model says that Department B has a larger mean attrition rate than department A. The model estimates that department A has an attrition rate of 9.1% and that department B has an attrition rate of 10.9% (the estimates brms provides are on the log odds scale, so you have to do some algebra to get these from the model output). From this data, we're fairly confident that the difference is real (the credible interval for Department B's effect on attrition excludes 0).

So, to your question, can a t-test be used on this data? Well, comparing it to a more complex model, which I would argue makes better assumptions about the data, yes a t-test can be used. The inferences are very similar, which is kind of surprising to me (I haven't compared estimates of uncertainty, which are likely to be different given the likelihoods are different). Anyway, I think this is a really good example of a) simple methods getting the job done, and b) double checking your inferences are correct by relaxing assumptions about the simple approach.

$\endgroup$
8
  • $\begingroup$ Thanks for the reply Demetri. I updated my question with some data (sorry for the table formatting). The question I’m trying to answer is do Dept A teams tend to attrit at a lower rate than Dept B teams. I’ll have to read up on hierarchical model. Would t test still be appropriate given this data? $\endgroup$
    – Insu Q
    Jul 10, 2019 at 2:08
  • $\begingroup$ If you post the data, I can run the hierarchal model and tell you. I can't parse a latex table, so if you could provide as a csv (via google sheets or something), that would be best $\endgroup$ Jul 10, 2019 at 2:09
  • $\begingroup$ I really appreciate the thorough explanation, this is really helpful! $\endgroup$
    – Insu Q
    Jul 10, 2019 at 2:50
  • $\begingroup$ @InsuQ One thing I forgot to mention is that the number of people in each team would be much more informative. If the teams in Department A have more than 100 people, and the teams in Department B have fewer than 20 people, then a t-test is inappropriately giving equal weight to all observations. $\endgroup$ Jul 10, 2019 at 12:55
  • $\begingroup$ What’s the best way to account for this? $\endgroup$
    – Insu Q
    Jul 10, 2019 at 12:57

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.