2
$\begingroup$

I'm wondering if a study size is relatively small and for whatever reason you insist on doing inferential statistics, would classical tests of hypothesis be more appropriate than regression analyses? The idea of predicting a change in y based on a change in x seems to me to require more data than to simply say with a fair amount of confidence that y and x are indeed different.

Am I right in this speculation or are they equally valid despite the N?

Note: I realize it depends on the research question but let's say you are designing a study and so you can pick the research question, or you realized your N is low and so you may consider changing "predict" to "associated" in the research question.

$\endgroup$
  • 1
    $\begingroup$ There is no meaningful difference between "predict" and "associated" in a regression setting: regression does not assess causation. Could you therefore explain the sense in which you are using "predict" in your question? $\endgroup$ – whuber Jan 14 at 15:55
  • $\begingroup$ I thought that was the difference. Prediction models can predict the relationship between x and y, while classical tests of hypothesis can only assert that they are associated but not predict their association? $\endgroup$ – Paze Jan 14 at 16:49
2
$\begingroup$

Many standard statistical tests can be reframed as simple linear models. So in your example, there is no difference between a two sample T-test testing for differences in X and Y versus a linear regression of Y and X against a grouping indicator. Your intuition that regression requires more data is not correct.

Also, as whuber mentions in his comment, you need to be careful with how you are using the word predict. It is thrown around carelessly in terms of regression models when most of the time it would be correct to say associated or explained instead.

This page gives a lot of handy information about the statistical tests to linear models correspondence.

https://lindeloev.github.io/tests-as-linear/

$\endgroup$
  • $\begingroup$ What if I wanted to ask "is there a difference in the mean of x and y, controlled for z"? As far as I understand, regression tells us the change in y per change in x, and tests of hypothesis usually only have 2 variables? $\endgroup$ – Paze Jan 14 at 16:48
  • $\begingroup$ Sorry, I clarified a notation in my answer that wasn't accurate. A T test between X and Y is equivalent to a regression model of W ~ 1 + Gx, where W is the values of both X and Y concatenated, and Gx is an indicator variable for being in group X. This regression model will yield results identical to the T test results. If you want to control for another variable, you would just add it to the regression: W ~ 1 + Gx + Z. $\endgroup$ – daenris Jan 14 at 16:55
  • $\begingroup$ Okay so the only scenario where I'd use a t-test over a regression is if I don't meet the criteria for a regression...? I am so confused with these two, they sound exactly the same only regressions can include confounders. $\endgroup$ – Paze Jan 14 at 16:57
  • $\begingroup$ Without confounders they are identical, so there is no difference between using a T test or a regression model as long as it is specified correctly. A T test can't directly handle additional confounding variables, so in the case that you have those you would need to use a regression model. Testing whether the mean is different between two groups (t test) is identical to testing whether there is a non-zero slope between the datapoints for the two groups (regression). $\endgroup$ – daenris Jan 14 at 17:00
  • $\begingroup$ So there is literally no reason to use a t test since the invention of the regression model? $\endgroup$ – Paze Jan 14 at 18:01

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.