3
$\begingroup$

Say I have two groups of patients and like to test the association between this grouping and another pathological feature on these patients. Given that the pathological features is nearly continous. I can do a t-test / rank-sum test. Alternatively, I can set a cutoff on the pathlogical feature and make a 2*2 contingency table and apply chi-squared test.

My question the difference between these to method. My initial guess is that I would loss power due to the lost of continous value in the 2*2 contigency table. Is t-test / rank-sum test always prefered due to the reason above? Or there is some deeper discussion on this matter?

Improve this question
$\endgroup$
3
  • 1
    $\begingroup$ Power is difficult to deal with definitively because of various assumptions involved. Best to begin by determining exactly what you want to know and test. Then to use the test for that. $\endgroup$
    – BruceET
    Commented Aug 2, 2019 at 4:25
  • $\begingroup$ To follow what @bruceet said, the appropriateness of your test for your analytic needs had higher precedence than the power. Determine what you need to do first then work out the power. $\endgroup$
    – ReneBt
    Commented Aug 2, 2019 at 4:45
  • $\begingroup$ If the variable you observe is truly the variable you want to test, then throwing away information can only decrease the power of your test and it is valid to be concerned about it. Agreed that power calculations and such should be left until you've identified the test (or never--I hate power calculations!!!). $\endgroup$
    – Sheridan Grant
    Commented Aug 12, 2019 at 6:32

3 Answers 3

Reset to default
1
$\begingroup$

I can set a cutoff on the pathlogical feature and make a 2*2 contingency table and apply chi-squared test.

This is perhaps the worst thing you could do. Binarizing a continuous outcome destroys information (quite literally, there are more bits of information in the continuous variable than there are in a binary variable). You also leave yourself open to residual confounding if the outcome in your (arbitrarily selected groups) are not completely flat.

I would list out all 10+ issues with this approach (as Frank Harrell does in his excellent course notes) but instead I will just point you to section 2.4.1 of the linked PDF. As for power, if I recall correctly you are best off if you split at the population median (which is unknown to you anyway) and even then the test has reduced power as compared to a more appropriate test.

Ok, that's enough harping on categorization.

Improve this answer
$\endgroup$
0
$\begingroup$

You are correct that discretizing your continuous data and using a chi-squared test is probably a bad idea because of the loss of information and ensuing decrease in the power of the test. If you have two groups and independent observations with a more-or-less continuous outcome, then the t test or rank-sum test are two good options, depending what you want to test and what assumptions you're willing to make.

You should read up on them more to decide what you need. If you have a reasonably large sample and simply want to test for a difference in means, the t test is often the best choice. Rank-sum tests the null hypothesis that $P(X>Y) = P(Y>X)$ where $X$ and $Y$ are the random variables representing the two groups. This is a bit of a bizarre hypothesis, but if you have reason to believe the means are similar yet the distributions are differently shaped, it could be a good choice. Kolmogorov-Smirnov or tests for dominance are also options, though they are more difficult to understand.

Improve this answer
$\endgroup$
-1
$\begingroup$

The power of test: t-test > rank-sum test > $2\times2$ table test

Requirements: t-test---strong (Normal, at least near symmetric); rank-sum test --- middle (random variable is continuous); $2\times2$ table test --- weak (following binomial distribution after cutoff)

So according to your confidence on data properties, select the most powerful test.

This principle is very useful for small sample size. When sample size is large, the power is high enough for 3 tests to detect the meaningful difference. When sample size is large, using t-test is always reasonable because of central limit theorem. For small sample size, power of test is important, also the validation of test heavily depends on data meeting the requirements.

Improve this answer
$\endgroup$
8
  • 2
    $\begingroup$ Your opening comment on the power only holds if the data are normally distributed. For non-normal data, it is possible for the rank sum test to have more power to reject than a t test. Moreover, these tests have different null hypotheses, and the CLT does not get you around the fact that what is being inferred is different for each test. Finally, discretizing continuous variables so that one can perform a $\chi^{2}$ test not only drops power, it adds bias (a bias which itself depends on the choice of cut-off). $\endgroup$
    – Alexis
    Commented Aug 2, 2019 at 4:00
  • $\begingroup$ Another aspect that is missing from the answer is potential heteroscedasticity, skewness, etc. The rank sum test strongly assumes that the data differ in mean and only in mean. If the distributions differ in anything else, it‘s hard to detect what the test actually detected. $\endgroup$
    – StoryTeller0815
    Commented Aug 2, 2019 at 4:39
  • $\begingroup$ @StoryTeller0815 That is not an accurate description of the assumptions underlying the rank sum test, which is far more general. The general form of the rank sum hypothesis is $H_{0}$: $P(X_{A} > X_{B}) = 0.5$ with $H_{A}$: $P(X_{A} > X_{B}) \ne 0.5$, and its assumptions are that $X$ is (a) continuous, and (b) that within $A$ and within $B$ $X$ is i.i.d. The additional assumptions the the distributions of $A$ and $B$ are (i) identically shaped, and with (ii) identical variance are stringent add-ons which make the rank sum test a test of location (e.g. mean, median) shift. $\endgroup$
    – Alexis
    Commented Aug 3, 2019 at 15:23
  • $\begingroup$ Thanks for clarification but I thought we are talking about the location test? And this is extremely sensitive to any differences in the distributions other than location shifts. $\endgroup$
    – StoryTeller0815
    Commented Aug 3, 2019 at 15:27
  • $\begingroup$ The t test does not assume normality of the data, a common misconception. It is very general, and often a good choice when you want to test for a difference in means. $\endgroup$
    – Sheridan Grant
    Commented Aug 8, 2019 at 7:32

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.