1
$\begingroup$

I want to calculate the p-value between subgroups of my samples. For that, I am using the T.TEST function of Excel. But I do not understand the last parameter, type:

  1. Paired
  2. Two-sample equal variance (homoscedastic)
  3. Two-sample unequal variance (heteroscedastic)

In my case, I cannot use paired (not the same size). But how can I determine if my variance is equal or not? Can I just calculate the variance of my sample and compare them? If, yes, what is the threshold of equality? Or is it a general assumption, like the tails: gaussian or not?

Also, can I use the t-test if the size of my data are unbalanced? Some of my subgroups have about the same size, but some others are 85%-15%.

[EDIT] To add more context. I have a questionnaire that people answered with Likert-scale questions (from 1 to 5). I split the data into different subgroups to make some analyses: gender, country of origin (Japanese or not)... I calculated the t-test p-value for each variable (questions) for each subgroup.

$\endgroup$
8
  • $\begingroup$ Hi @Dark Patate and welcome to CV ! A first step could be to simply plot both your subgroups in a conditional boxplot to visually compare the variances. Small differences in variance are fine but if a group's variance is 3-4 times higher than the other's, that'll likely be problematic. Also, you might want to look at: stats.stackexchange.com/questions/45666/… $\endgroup$
    – Fanfoué
    Aug 1, 2022 at 9:08
  • $\begingroup$ Thanks! I checked the variance of my groups (I didn't do it for all yet), and even if most of the group's variances are 1-2 times bigger, some of them are bigger. I have one 4 times. Can I use two different versions of the t-test for different variables? Also, after reading the post you shared, maybe the t-test is not the best one. I want to check if it is statistically significant to analyse my 2 subgroups. I am not that good with statistics, so everything is mixing up in my head ^^' $\endgroup$ Aug 1, 2022 at 10:02
  • $\begingroup$ Some statistical people regard the unequal variant as the default for use. It seems to behave well if the variances are close to equal, but the opposite isn't quite so true. $\endgroup$
    – Nick Cox
    Aug 1, 2022 at 11:58
  • $\begingroup$ @DarkPatate Given the information you gave us here, I think you could compare your means with a Welsh's ANOVA. However, I am not a statistician so you'd better wait for more expert opinions. You could also look at this answer: stats.stackexchange.com/a/232098/203941 $\endgroup$
    – Fanfoué
    Aug 1, 2022 at 14:21
  • $\begingroup$ Indeed, after reading more about Welch's ANOVA, it looks like it's a good option for me since my data doesn't have the same variance and size. What's the difference between Welch's ANOVA and t-test with the heteroscedastic parameter? $\endgroup$ Aug 2, 2022 at 3:06

1 Answer 1

0
$\begingroup$

I guess, the underlying assumption is that both sample groups come from normal distributions. In this case, the sample variance from either group should follow $\chi^2$ distribution, whilst the actual variances should follow inverse $\chi^2$. You could construct a test that extracts variance from the first group and then tests how well the data from the second group follows it (i.e. is the second sample variance likely, given your guess based on the first one). Not sure whether there is a name for this procedure. Also, it is a good idea to know how much difference you want to be sensitive to. For example, do you care about 0.1% difference in variance?

I am sure there must be a software package that does this anyway, but since this is not something I need to do often, i prefer more manual route :)


ADDENDUM (keeping the previous text for comments to make sense).

How can we test whether two normal distributions have the same or similar variance, given two groups of samples. I get the feeling this is more detail than what you are after, but since there are now other answers, it makes sense to suggest a viable route to go

Firstly, you need to have some sense of what is a significant difference. This is subject matter expertise. One way to start may be to estimate the sample variance of either group and take a fraction of that. For example:

\begin{align} \{X_1,\,X_2,\dots X_N\},\quad X_{1\dots N}\sim N\left(\mu_1,\sigma_X^2\right) \\ \{Y_1,\,Y_2,\dots Y_M\},\quad Y_{1\dots M}\sim N\left(\mu_2,\sigma_Y^2\right) \end{align}

Let sample variances be:

\begin{align} \bar{S}^2_X=\frac{\sum_{i=1\dots N}\left(X_i-\bar{X}\right)^2}{N-1},\quad \bar{X}=\frac{\sum_{i=1\dots N}X_i}{N} \\ \bar{S}^2_Y=\frac{\sum_{i=1\dots M}\left(Y_i-\bar{Y}\right)^2}{M-1},\quad \bar{Y}=\frac{\sum_{i=1\dots M}Y_i}{M} \end{align}

You may then ask whether $\left|\sigma^2_X-\sigma^2_Y\right|\le MIN\left(\bar{S}^2_X,\,\bar{S}^2_Y\right)/10$. Or something similar. This gives you your null hypothesis.

Next you could use the fact that, by Student's Theorem the ratio of sample variance and actual variance is chi-2 distributed

$$ \frac{\left(N-1\right)\bar{S}^2_X}{\sigma^2_X}\sim\chi^2\left(N-1\right) $$

Then the distribution of the variance, given data, is given by scaled-inverse-chi2:

$$ \sigma^2_{X}\sim Scaled-Inv-\chi^2\left(N-1,\,\bar{S}^2_X\right) $$

And similar for $\sigma^2_Y$. So now you could actually compute the probability $\left|\sigma^2_X-\sigma^2_Y\right|\le MIN\left(\bar{S}^2_X,\,\bar{S}^2_Y\right)/10$ and do a proper hypothesis test of whether variances are the same.

$\endgroup$
2
  • $\begingroup$ Yes, I made the assumption that everything is normal. Maybe it's not a good assumption. By "extracting the variance", it's different from just calculating it (with a specific function or SD^2)? $\endgroup$ Aug 1, 2022 at 10:09
  • $\begingroup$ The specific function will only give you sample variance, @DarkPatate. I have extended the answer. Either way, from the comments above it would seem you have settled for heteroschedastic test. $\endgroup$
    – Cryo
    Aug 2, 2022 at 13:05

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.