I am studying Statistics for data science since few months..

1)I am learning that, when we have to compare multiple samples (>2) then a T test would be tedious and instead we go for ANOVA and conduct a 'F Test'.

2)Above understanding kind of creates a 'mutually exclusive requirement between F test and T test."

3)I have also learned that, the T test ( be it : 1 sample/paired/2 sample ) basically tests for differences in means whereas 'F test' tests for differences in Variances.

4)Now, suppose two samples groups are having nearly equal means but highly different variances, then, both tests would give different answers right?

T test would say 'they are not different'. But 'F test' would say 'they are different'.

Or even for a reverse case. (hugely different means, but nearly same variances)..

5)So based on what, (the mean? or the variance?) we are finally going to decide their true difference?

6)So Question is: How are they related? If original objective was to find out two or more samples are different or no, then how 'looking for means'(i.e. choosing the T test) for smaller no of sample groups, gets changed to 'looking for variances' when no of sample groups are >2? (When the fact is :the variance and mean are basically independent characteristics of a sample group)

7)Should Not these both metrics be checked for finding whether truly the two samples are different or Not ?

( I have mentioned serial numbers to points that I have stated. Kindly point out if any of them is a basically wrong understanding. Would appreciate if answers are given for each point)

  • $\begingroup$ What do you exactly mean by "comparing samples"? Are you talking about comparing whether the mean of the population they come is the same/different? Or are you talking about checking whether their distribution is the same/different? $\endgroup$ – David Jun 5 '19 at 14:44
  • $\begingroup$ I am not sure !! Because thats what I want to know.! Should we not look for both for deciding "those two sample groups are different or not" in all aspects ? I could not find any tutorial which highlighted this view.. Most of the tutorials explain like "...for comparing more than two groups, go for F test.. .. ". That time the view point changes from "looking at mean" to "looking at variances !!".. Hence I am not clear on this! $\endgroup$ – Abhijeet Kelkar Jun 5 '19 at 14:55
  • $\begingroup$ As a new student of stat, i dont know what to look out for !.. most of tutorials say.. " T test OR F test".. none of the tutorial said " check for both T AND F !! ( my opinion : Shouldn't we look from both angles ? (i.e. the means as well as the variances)? $\endgroup$ – Abhijeet Kelkar Jun 5 '19 at 15:00
  • $\begingroup$ The below link kind of goes there: I have referred it already. But not exactly it answers my question ): stats.stackexchange.com/questions/78150/… $\endgroup$ – Abhijeet Kelkar Jun 5 '19 at 15:04
  • 1
    $\begingroup$ Well, making a "test" is finding the answer to a question. The first thing you need to know is what the actual question is! $\endgroup$ – David Jun 5 '19 at 15:45

The terms t-test and F-Test are ambiguous, because any test where the test statistic has a t-distribution (under the null hypothesis) is called t-test and any test where the test statistic has an F-distribution is called F-test. There are more than one instances of these.

This is relevant to your question because there is an F-test that compares the variances of two samples, but this is not the F-test used in standard ANOVA-analysis. In fact the ANOVA F-test compares between-group and within-group variability, and between-group variability is in fact measured by squaring and summing up differences between group means, so in this setup both t- and F-tests are about comparing group means. In fact, if you have only two groups/factor levels, the F-test statistic is the square of the t-test statistic, and the F-test is equivalent to the two-sided t-test. For more than two groups the issue with t-tests is that the t-test can only compare two groups at one time, meaning that you will need several t-tests to compare all groups, involving issues with multiple testing (i.e., if you test several hypotheses at 5% level, the probability to find at least one wrong significance assuming that the null hypotheses are all true can be substantially higher than 5%).

Additionally, you are right that one may be interested in exploring both differences between means and differences between variances, and groups with same mean may still have different variances. You may indeed check them both, though this again involves multiple testing; there's no free lunch. In many applications of ANOVA it is either fairly reasonable to assume equal variances, or only mean differences are of substantial interest (e.g., only wondering whether one group performs "better" than another), therefore differences in variances are often not explicitly investigated (I will abstain from a statement about whether this would be "good" or "correct"; or rather my answer would be "it depends"...).

| cite | improve this answer | |
  • $\begingroup$ Thanks for the explanation $\endgroup$ – Abhijeet Kelkar Jun 6 '19 at 8:46

If you are comparing more than two groups and are interested in comparing their means then it is usual to do ANOVA as you say which tests the hypothesis that all group means are equal. Doing multiple $t$-tests is not quite equivalent because each test only tets if the means in those two groups are equal. Your point 1)

The use of the $F$ test to compare variances is used because what you compare in ANOVA is the variance between the group means versus the variance within groups. (Your point 3)

The remainder of your questions are hard to answer because, see my points above, I think you have some mis-conceptions about just what is going on.

| cite | improve this answer | |

Consider this formula

Ho: group1 and group2 has the same average
(e.g. do they have the same average height)
t = (mean-k)/(s/sqrt(n)), basic assumption. variance is known.

Ho: Different level of fertilizer (NPK) has no significant effect on plants.
F = n(mean-k)^2 / s^2, w/c is simply t^2

  1. from Practicality point of view this could be true correct.

2.If you have a control and treated group form the same population then they will be the same. But say if you have boys vs girls, location1 vs location2, they could be different.

  1. Correct.
  2. Possibly
  3. Depending on your objective. If you simply want to know if the group have different characteristics (like average) then t-test. If you want to know if certain applied factors (like different levels of cigarette's nicotine) have significant effects then use F-test.
  4. The formula is related but the application differs depending on your goal.

  5. No. since it doesn't makes any sense since t and F test have different goal or problem they're solving.

| cite | improve this answer | |

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.