Significance testing on weight of people from 2 populations given their height

Suppose we have $N_A$ and $N_B$ samples from population A and B respectively. For each sample, we have his/her weight and height. What interests us is whether there are a significant difference in weight between A and B. A t-test can do this easily. But we also know there is strong correlation between height and weight. If weight is not influenced by population but the height of samples are different between A and B, then we may reject $H_0$ by mistake. Is there a systematic way to correct samples' weight using their height so we can prevent its interference?

-
You say that there are two populations. Are these two pre-existing populations from which you've drawn random samples or are these two hypothetical populations to which you've applied some treatment through random sampling? – John Nov 17 '11 at 14:25
Pre-existing populations (two countries). – cogitovita Nov 18 '11 at 16:06
ANCOVA is not recommended. You need a true experiment for ANCOVA. – John Nov 18 '11 at 21:30

This kind of task is solved by ANCOVA (analysis of covariance). According to its model, weight is dependent on two effects (apart from constant), the group effect and the covariate (height) effect: $weight=constant+group+height$. Here group effect is clean, in the sense that the possible difference the two groups in average height is washed out. So you may safely rely on significance of group effect. Before you do the above analysis you should make sure that the strength of dependency of weight on height doesn't differ in the two groups. To do it, try the model with the interaction term: $weight=constant+group+height+group*height$. If the interaction is nonsignificant you can turn to the above two-effect model (while if it is significant you should apply a nested model that is a bit more complex).

The above simple approach however assumes that weight is dependent on height linearly, and we know that it is certainly not true. Another nasty thing is that weight depends on height heteroscedastically, that is, variation of weight is larger for big heights than for small heights. What to do? One way is to transform weight prior to the analysis so that weight by height scatter-cloud is about linear and homoscedastic. Another opportunity is to try generalized linear model instead of classic ANCOVA. That procedure offers various link functions which in fact perform the transformation for you implicitly.

-
+1 for pointing out that the covariate and predictor need to be unrelated. I think though, that it's important to clarify that if it's a quasi experiment, as implied by current wording, there will typically be some relationship between the covariate and predictor and, even if the interaction is not significant, ANCOVA should be avoided. – John Nov 17 '11 at 14:31

I'm not certain of your question or your experimental design but taken on it's face, do a t-test of the heights.

If your question is, "if there really was no difference in heights would there be a difference in weights?", you're out of luck because there is no way to know that.

Other answerers have recommended ANCOVA, which is a method to increase power if there is no relationship between the predictor and covariate. Essentially, you remove some response variability because you know it belongs to somethign else. But, you want to know if there is a relationship between your predictor and covariate, which is a different question. And, if it is true, precludes ANCOVA (Porter & Raudenbush, 1987). (further to that, ttnphns is correct that you must assess whether there is any relationship between height and group. However, I believe that significance is too stringent a cutoff given there is supposed to be NO relationship. One could have a non-significant amount of shared variance between height and group and then still have the removal of that variance spuriously generate a significant difference in weight.)

Do a t-test with height as the response and group as the predictor for starters. Then get standardized effect sizes from both tests. If they're similar you're basically stuck. If weight is substantially larger you could make a cogent argument for your predictor influencing weight. Perhaps if you come back with another question at that point you could get more targeted help... flesh out the design of your study better in your question at that time.

-

First of all, if you have reason to assume there are generally no differences between the heights of the two groups and if you have done proper randomization, your sample sizes are large, etc, you can argue that you have randomized over the heights and your conclusion can remain as is: it can be shown that if you've heeded some rules, you'll get unbiased estimates and trustworthy confidence intervals, albeit not the most efficient (read: most narrow) ones, so the method will be less powerful in general. This statement is only valid if your assumptions hold. Note: this is also what you do by e.g. ignoring gender in your comparison between the populations: you assume the two genders are properly divided over both populations and consequently represented in your samples. (while re-reading, I realize that this is probably confusing if you two populations happen to be men and women - if so, replace gender by adulthood in the previous sentence)

Since you have the height data at hand, though: no reason not to use it, and it should only increase the stability and efficiency of the results. So how do you go about it?

I hardly ever look at a t-test as such. In fact, two-sided t-tests are equivalent to F-tests from linear models: in your example without the height, you essentially compare the performance of the following 2 models (groupB is an artificial or dummy variable that is zero for observations from population A and 1 for observations from population B): $$weight = \beta_0 + \epsilon$$ and $$weight = \beta_0 + \beta_1 * groupB + \epsilon$$ where $\epsilon$ is standard normal: if there is a difference between the two groups, the second model will be necessary and perform much better than the first one. A little thinking (or a little reading up) shows you that $\beta_0$ in the first model represents the mean weight over the two populations, while in the second model, $\beta_0$ is the mean in population A and $\beta_1$ is the mean difference in weight between the two populations, so this test is already intuitively equivalent to testing whether the mean difference between the groups is zero, which is what a t-test does. This intuition can be confirmed rigorously through maths and is in just about any textbook on regression.

You compare the 'performance' of these models by an F-test or likelihood ratio test (simply put: a transformed version of the ratio of the maximum likelihoods of both models is known to follow (under the assumption that the simpler model performs well enough) a well-known distribution, so you can find out how 'reasonable' this result is)

The good news is that you can also do this for more complex models, that account for other effects. If you want to correct for the effect of height, then what you really want to do is compare the following two models: $$weight = \beta_0 + \beta_2 * height + \epsilon$$ and $$weight = \beta_0 + \beta_1 * groupB + \beta_2 * height + \epsilon$$ Following the logic of before, you can see that $\beta_1$ in the second model now represents the mean difference in weight between the two groups for any given height. So comparing these two models is the same as checking whether there is a height-corrected difference in weights between the groups.

So: you feed these two models to a likelihood ratio test and you're done.

How to do that in practice depends on the software you have available (e.g. R has lm to fit linear models and aov or anova to perform the likelihood ratio test --- note the name ANOVA refers to a methodology that is also statistically equivalent to linear regression as well, so in some textbooks (mostly depending on the age of the book and the field the book stems from) you will find more on ANOVA)

-
I'm not seeing how this will help determine if the difference in weights is determined by the difference in heights. – John Nov 17 '11 at 15:33
@John : from the OP's question: "What interests us is whether there are a significant difference in weight between A and B". This test will help answer that question. Other model comparisons will answer other questions (which may require to include the interaction term). – Nick Sabbe Nov 17 '11 at 16:16
No, this will not tell you that if height is correlated with group... which is the the problem the OP is worried about. – John Nov 17 '11 at 18:35
It will neither if his sample sizes are too small or if he had an unlucky sample, and a hidden covariate may screw things up, as does interaction. Having a nonsignificant p-value never does ensure you that the null hypothesis is true, though. Either way: my answer should have been more careful in that regard, so my bad. Thanks for the correction. – Nick Sabbe Nov 17 '11 at 19:55