4
$\begingroup$

I am investigating whether the value of a biological marker (LRG) is significantly different in two groups (Anemic, Non-Anemic). I tried with t test which gives a p-value = 0.1. I also tried it with simple linear regression which gives me a p-value < 0.05. Why do these two methods contradict?

Please find the sample data below:

structure(list(LRG = c(29.943655, 33.709029, 50.130356, 44.899403,
44.908902, 29.563658, 40.387807, 44.074283, 26.804545, 33.243105,
43.487849, 47.765206, 30.877019, 45.593778, 45.755752, 47.765206,
29.283398, 32.477029, 41.079931, 25.466288, 42.42234, 45.774815,
55.92435, 40.986301, 29.645058, 13.06562, 36.435235, 35.763044,
41.895857, 46.328249, 39.036493, 51.697295, 60.231314, 30.768168,
56.92095, 24.471153, 42.168373, 51.746123, 39.865164, 49.578044,
31.885332, 46.165911, 35.634292, 39.017898, 41.342265, 32.103695,
24.041242, 64.768799, 24.981991, 29.202066, 28.227211, 47.467557,
35.257528, 23.074732, 47.151096, 46.376016, 25.493204, 47.707569,
38.330724, 37.478607, 34.96377, 62.165858, 38.098931, 39.529684,
31.167436, 45.232089, 29.717426, 38.210169, 52.656175, 36.131191,
40.892702, 36.352285, 32.44059, 56.881002, 54.744048, 55.665948,
62.606058, 47.160679, 44.225834, 35.910252, 45.746221, 46.060923,
50.033367, 30.052288, 26.948435, 35.395318, 25.286791, 24.671939,
37.363321, 32.637409, 21.223423, 43.165211, 45.210506, 51.115379,
23.049126, 25.361838, 33.460265, 27.180943, 29.840756, 32.223067,
31.495611, 31.870592, 22.049812, 27.680767, 26.903608, 26.308697,
19.677732, 23.086453, 34.417005, 28.270485, 20.197548, 29.14948,
22.303185, 13.486439, 27.703429, 28.793045, 54.365346, 30.449766,
32.552956, 16.874961, 13.53843, 42.636807, 21.632747, 19.113716,
21.655082, 26.180807, 13.894822, 10.069526, 17.275089, 38.374135,
21.015146, 23.489769, 20.621091, 32.476203, 29.119129, 13.315579,
20.784633, 25.872546, 24.851799, 29.954852, 21.662527, 32.960105,
27.424035, 19.714851, 2.214312, 30.259318, 23.355292, 33.938206,
17.57151, 22.236104, 29.339238, 11.551196, 21.841241, 23.953275,
17.045382, 24.035561, 46.940455, 57.887882, 41.368861, 37.613676,
22.079615, 23.101384, 30.327866, 19.084041, 21.543416, 21.312706,
26.632387, 23.415055, 20.011861, 29.88639, 24.761859, 34.943143,
23.250725, 29.5934, 36.786986, 18.16479, 4.948129, 39.088652,
17.899953, 34.847586, 26.845274, 37.661072, 25.695198, 24.362309,
12.956978, 34.908524, 17.535549, 18.64102, 18.84302, 28.943873,
31.866202, 22.530572, 22.323189, 22.851646, 36.561401, 25.195236,
30.532669, 40.150547, 61.188579, 36.479437, 14.934046, 25.791526,
46.533768, 23.277663, 24.524714, 17.36282, 21.150446, 28.747536,
16.313392, 17.517356, 31.009189, 19.589384, 31.208123, 38.280472,
26.225723, 30.2553, 19.367809, 15.014214, 25.195236, 20.182012,
41.901452, 29.968493, 38.953723, 32.376394, 31.47704, 32.827774,
17.144973, 18.247091, 20.786502, 21.515321, 26.670785, 26.728928,
30.42365, 22.634373, 22.464555, 15.66651, 24.152391, 15.237195,
41.711997, 30.512842, 22.776037, 30.493018, 19.783559, 17.9182,
21.318734, 32.687223, 19.598624, 27.223976, 29.061784, 22.360874,
28.571029, 21.356159, 19.266374, 24.840458, 21.78724, 23.552849,
18.567642, 14.853933, 31.258095, 23.928957, 16.716953, 15.651969,
27.334131, 23.159622, 21.082857, 25.073001, 20.667487, 30.966137,
34.178797, 32.779891, 30.0303, 27.902494, 26.037788, 38.963068,
25.663705, 21.444966, 24.409059, 19.550525, 29.829776, 36.992054,
30.598266, 32.192221, 29.733034, 27.224715, 24.822152, 36.53462,
25.57536, 27.785977, 30.161774, 20.152162, 24.023566, 28.926129,
25.48025, 23.814114, 25.290128, 17.642658, 30.438813, 31.522585,
34.016624, 32.716848, 43.607187, 32.003674, 28.527149, 26.739744,
54.704873, 33.263861, 38.948606, 19.764359, 30.882791, 16.131174,
29.836688, 30.424953, 44.627006, 17.76259, 37.888669, 27.895639,
26.24205, 38.349396, 30.134089, 39.956491, 31.36246, 28.91236,
23.814114, 17.935853, 24.273742, 23.442848, 36.827538, 22.701574,
29.24304, 20.607224, 20.366253, 24.605375, 31.452951, 30.848074,
26.842127, 36.748906, 31.049514, 21.425458, 17.623163, 41.112452,
29.599919, 33.613293, 25.692515, 47.806569, 30.770139, 25.169942,
29.503465, 33.658354, 20.545482, 23.534762, 19.072565, 33.008328,
25.38289, 21.171503, 24.57589, 31.792554, 22.666466, 14.869722,
27.69591, 42.675456, 31.477446, 36.474968, 21.900282, 16.761553,
22.737812, 34.089721, 34.843367, 35.610442, 44.459678, 24.808415,
26.195383, 36.061953, 24.750292, 41.236014, 34.592096, 35.726523,
43.197581, 21.815773, 35.15915, 34.089721, 37.476127, 24.013556,
30.140031, 32.487199, 27.090801, 31.715382, 30.210758, 20.669488,
36.565342, 18.849212, 43.204111, 31.335977, 17.543802, 30.249336,
13.312084, 25.492565, 19.728252, 47.595315, 28.281454, 43.328204,
22.257598, 19.944246, 24.181672, 26.330717, 28.783194, 34.141237,
59.368021, 23.741866, 39.521679, 21.789766, 29.484174), hbcat_all = c("Not-anemic",
"Not-anemic", "Not-anemic", "Not-anemic", "Not-anemic", "Not-anemic",
"Not-anemic", "Not-anemic", "Not-anemic", "Not-anemic", "Not-anemic",
"Not-anemic", "Anemic", "Not-anemic", "Not-anemic", "Not-anemic",
"Not-anemic", "Not-anemic", "Anemic", "Not-anemic", "Not-anemic",
"Anemic", "Not-anemic", "Not-anemic", "Not-anemic", "Not-anemic",
"Not-anemic", "Anemic", "Not-anemic", "Not-anemic", "Anemic",
"Anemic", "Not-anemic", "Not-anemic", "Anemic", "Not-anemic",
"Not-anemic", "Not-anemic", "Not-anemic", "Not-anemic", "Not-anemic",
"Not-anemic", "Not-anemic", "Not-anemic", "Not-anemic", "Not-anemic",
"Not-anemic", "Not-anemic", "Anemic", "Not-anemic", "Not-anemic",
"Not-anemic", "Not-anemic", "Not-anemic", "Anemic", "Not-anemic",
"Not-anemic", "Not-anemic", "Not-anemic", "Not-anemic", "Not-anemic",
"Not-anemic", "Not-anemic", "Not-anemic", "Not-anemic", "Not-anemic",
"Not-anemic", "Not-anemic", "Not-anemic", "Not-anemic", "Not-anemic",
"Not-anemic", "Not-anemic", "Not-anemic", "Not-anemic", "Not-anemic",
"Anemic", "Not-anemic", "Not-anemic", "Not-anemic", "Not-anemic",
"Not-anemic", "Not-anemic", "Not-anemic", "Not-anemic", "Not-anemic",
"Not-anemic", "Not-anemic", "Not-anemic", "Anemic", "Not-anemic",
"Anemic", "Not-anemic", "Not-anemic", "Not-anemic", "Not-anemic",
"Not-anemic", "Not-anemic", "Not-anemic", "Not-anemic", "Not-anemic",
"Not-anemic", "Not-anemic", "Not-anemic", "Not-anemic", "Not-anemic",
"Not-anemic", "Not-anemic", "Not-anemic", "Not-anemic", "Not-anemic",
"Not-anemic", "Anemic", "Not-anemic", "Not-anemic", "Not-anemic",
"Not-anemic", "Not-anemic", "Not-anemic", "Not-anemic", "Not-anemic",
"Anemic", "Not-anemic", "Not-anemic", "Not-anemic", "Not-anemic",
"Anemic", "Not-anemic", "Not-anemic", "Not-anemic", "Not-anemic",
"Not-anemic", "Not-anemic", "Not-anemic", "Not-anemic", "Not-anemic",
"Not-anemic", "Not-anemic", "Not-anemic", "Not-anemic", "Not-anemic",
"Not-anemic", "Not-anemic", "Not-anemic", "Not-anemic", "Not-anemic",
"Not-anemic", "Not-anemic", "Not-anemic", "Not-anemic", "Not-anemic",
"Not-anemic", "Not-anemic", "Not-anemic", "Not-anemic", "Not-anemic",
"Not-anemic", "Not-anemic", "Not-anemic", "Not-anemic", "Not-anemic",
"Not-anemic", "Not-anemic", "Not-anemic", "Not-anemic", "Not-anemic",
"Not-anemic", "Not-anemic", "Not-anemic", "Not-anemic", "Anemic",
"Anemic", "Not-anemic", "Not-anemic", "Not-anemic", "Not-anemic",
"Not-anemic", "Not-anemic", "Not-anemic", "Not-anemic", "Not-anemic",
"Not-anemic", "Not-anemic", "Anemic", "Anemic", "Not-anemic",
"Not-anemic", "Not-anemic", "Not-anemic", "Not-anemic", "Not-anemic",
"Not-anemic", "Not-anemic", "Not-anemic", "Not-anemic", "Not-anemic",
"Not-anemic", "Not-anemic", "Not-anemic", "Not-anemic", "Anemic",
"Not-anemic", "Not-anemic", "Not-anemic", "Not-anemic", "Not-anemic",
"Not-anemic", "Not-anemic", "Not-anemic", "Not-anemic", "Not-anemic",
"Not-anemic", "Not-anemic", "Not-anemic", "Anemic", "Anemic",
"Not-anemic", "Not-anemic", "Not-anemic", "Not-anemic", "Not-anemic",
"Not-anemic", "Not-anemic", "Not-anemic", "Not-anemic", "Not-anemic",
"Not-anemic", "Not-anemic", "Not-anemic", "Anemic", "Not-anemic",
"Not-anemic", "Not-anemic", "Not-anemic", "Not-anemic", "Not-anemic",
"Not-anemic", "Not-anemic", "Not-anemic", "Not-anemic", "Not-anemic",
"Not-anemic", "Not-anemic", "Not-anemic", "Not-anemic", "Not-anemic",
"Not-anemic", "Not-anemic", "Not-anemic", "Not-anemic", "Not-anemic",
"Not-anemic", "Not-anemic", "Not-anemic", "Not-anemic", "Not-anemic",
"Not-anemic", "Not-anemic", "Not-anemic", "Not-anemic", "Not-anemic",
"Not-anemic", "Not-anemic", "Not-anemic", "Not-anemic", "Not-anemic",
"Not-anemic", "Not-anemic", "Not-anemic", "Not-anemic", "Not-anemic",
"Not-anemic", "Not-anemic", "Not-anemic", "Not-anemic", "Anemic",
"Not-anemic", "Not-anemic", "Not-anemic", "Not-anemic", "Not-anemic",
"Not-anemic", "Not-anemic", "Not-anemic", "Not-anemic", "Not-anemic",
"Not-anemic", "Not-anemic", "Not-anemic", "Not-anemic", "Not-anemic",
"Not-anemic", "Not-anemic", "Not-anemic", "Not-anemic", "Not-anemic",
"Not-anemic", "Not-anemic", "Not-anemic", "Not-anemic", "Not-anemic",
"Not-anemic", "Not-anemic", "Anemic", "Not-anemic", "Not-anemic",
"Not-anemic", "Not-anemic", "Not-anemic", "Not-anemic", "Not-anemic",
"Not-anemic", "Not-anemic", "Not-anemic", "Not-anemic", "Not-anemic",
"Not-anemic", "Not-anemic", "Not-anemic", "Not-anemic", "Not-anemic",
"Anemic", "Not-anemic", "Not-anemic", "Not-anemic", "Not-anemic",
"Not-anemic", "Not-anemic", "Not-anemic", "Not-anemic", "Not-anemic",
"Not-anemic", "Not-anemic", "Not-anemic", "Not-anemic", "Anemic",
"Anemic", "Not-anemic", "Not-anemic", "Not-anemic", "Not-anemic",
"Not-anemic", "Not-anemic", "Not-anemic", "Not-anemic", "Not-anemic",
"Not-anemic", "Not-anemic", "Not-anemic", "Not-anemic", "Not-anemic",
"Not-anemic", "Not-anemic", "Not-anemic", "Not-anemic", "Not-anemic",
"Not-anemic", "Not-anemic", "Not-anemic", "Not-anemic", "Not-anemic",
"Not-anemic", "Not-anemic", "Not-anemic", "Not-anemic", "Not-anemic",
"Not-anemic", "Not-anemic", "Not-anemic", "Not-anemic", "Not-anemic",
"Not-anemic", "Not-anemic", "Not-anemic", "Not-anemic", "Not-anemic",
"Not-anemic", "Not-anemic", "Not-anemic", "Not-anemic", "Not-anemic",
"Not-anemic", "Not-anemic", "Not-anemic", "Not-anemic", "Not-anemic",
"Not-anemic", "Not-anemic", "Not-anemic", "Not-anemic", "Anemic",
"Not-anemic", "Not-anemic", "Anemic", "Not-anemic", "Not-anemic",
"Not-anemic", "Not-anemic", "Not-anemic", "Not-anemic", "Not-anemic",
"Not-anemic", "Anemic", "Not-anemic", "Not-anemic", "Not-anemic",
"Anemic", "Not-anemic", "Not-anemic", "Not-anemic", "Not-anemic"
)), row.names = c(NA, -411L), class = c("tbl_df", "tbl", "data.frame"
))
Improve this question
$\endgroup$
1
  • $\begingroup$ What is "LRG"? LRG1? Or something else? $\endgroup$
    – Peter Mortensen
    Apr 21, 2022 at 21:38

2 Answers 2

Reset to default
12
$\begingroup$

You may want to double check this - T-test and linear model (or anova) give the same result if in the t-test you assume that the two groups have the same variance. This is what the linear model assumes. Using your data (in dat):

x <- dat[dat$hbcat_all == 'Not-anemic', 'LRG']
y <- dat[dat$hbcat_all == 'Anemic', 'LRG']

t.test(x, y, var.equal= TRUE)

    Two Sample t-test

data:  x and y
t = -2.0311, df = 409, p-value = 0.04289    <<<<<
alternative hypothesis: true difference in means is not equal to 0
95 percent confidence interval:
 -7.5833219 -0.1239241
sample estimates:
mean of x mean of y 
 29.97073  33.82436

Now linear model:

fit <- lm(data= dat, LRG ~ hbcat_all)
summary(fit)

Call:
lm(formula = LRG ~ hbcat_all, data = dat)

Residuals:
    Min      1Q  Median      3Q     Max 
-27.756  -7.388  -0.909   5.848  34.798 

Coefficients:
                    Estimate Std. Error t value Pr(>|t|)    
(Intercept)           33.824      1.822  18.565   <2e-16 ***
hbcat_allNot-anemic   -3.854      1.897  -2.031   0.0429 *  <<<<<
---
Signif. codes:  0 ‘***’ 0.001 ‘**’ 0.01 ‘*’ 0.05 ‘.’ 0.1 ‘ ’ 1

Residual standard error: 10.31 on 409 degrees of freedom
Multiple R-squared:  0.009986,  Adjusted R-squared:  0.007565 
F-statistic: 4.125 on 1 and 409 DF,  p-value: 0.04289

R default for t-test is unequal variance, hence the discrepancy you observe:

t.test(x, y, var.equal= FALSE)

    Welch Two Sample t-test

data:  x and y
t = -1.65, df = 34.27, p-value = 0.1081    <<<<<
alternative hypothesis: true difference in means is not equal to 0
95 percent confidence interval:
 -8.5985063  0.8912604
sample estimates:
mean of x mean of y 
 29.97073  33.82436 
```
Improve this answer
$\endgroup$
3
  • $\begingroup$ When I checked the variance of two groups using var.test in R I get the p-value < 0.05. So I should go with the t.test with var.equal= FALSE. is that correct? kindly advice. $\endgroup$
    – arshad
    Apr 21, 2022 at 8:27
  • 4
    $\begingroup$ var.equal= FALSE is more conservative and probably preferable. But ultimately it depends on what you want to do with these results and what you know or you want to assume about the Anemic/non-anemic groups. The data alone provides only weak evidence against $H_0$ even with the assumption of equal variance but perhaps you have some additional information that suggests that the difference is genuine and reasonable. I mean, p-values and CIs are useful but they are not the end of the story. $\endgroup$
    – dariober
    Apr 21, 2022 at 8:42
  • 4
    $\begingroup$ var.equal=FALSE is not more conservative in general. It is more conservative when the smaller group has the larger variance and less conservative when the larger group has the larger variance. $\endgroup$
    – Thomas Lumley
    Apr 22, 2022 at 0:11
6
$\begingroup$

Two-stage approaches (i.e., deciding whether to treat variance as (un)equal)

Following up a little bit on the discussion in the comments (i.e., using var.test() to decide whether to use var.equal = FALSE or not). As a general principle, using statistical tests to decide which test to use (a two-stage approach) works surprisingly poorly (e.g. Campbell and Dean 2014; Campbell 2021); it's best to decide in advance on principled grounds. Ruxton (2006) says

The problem with this flexible approach [i.e., using a test of homogeneity of variance to decide which version of the t-test to use] is that the combination of this preliminary test plus whichever of the subsequent tests is ultimately used controls Type I error rates less well than simply always performing an unequal variance t-test on every occasion (Gans 1992; Moser and Stevens 1992), this is one reason why it is generally unwise to decide whether to perform one statistical test on the basis of the outcome of another (Zimmerman 2004 and references therein). There are further reasons for not recommending preliminary tests of variances (e.g., Markowski CA and Markowski EP 1990; Quinn and Keough 2002, p. 42). Hence, I suggest avoiding preliminary tests and adopting the unequal variance t-test unless an argument based on logical, physical, or biological grounds can be made as to why the variances are very likely to be identical for the 2 populations under investigation.

Box-Cox transformation

Another approach that would be reasonable if you had decided on it in advance would be to do a Box-Cox procedure to decide how to transform the data:

m1 <- lm(LRG ~ hbcat_all, data = dff)
png("boxcox2.png")
par(las=1, bty = "l")
MASS::boxcox(m1)
dev.off()

box-cox plot showing max near lambda = 1/2

This shows that the optimal response transformation is near lambda = 1/2, i.e. taking the square root of the response.

summary(m2 <- lm(sqrt(LRG) ~ hbcat_all, data = dff))
Coefficients:
                    Estimate Std. Error t value Pr(>|t|)    
(Intercept)           5.7122     0.1651  34.599   <2e-16 ***
hbcat_allNot-anemic  -0.3150     0.1719  -1.832   0.0677 .

With this model the effect looks not-quite-significant at the p=0.05 level.

Furthermore, the model diagnostics look OK.

par(las=1, bty = "l")
par(mfrow=c(2,2))
plot(m2)

diagnostic plots

In this simple a model the diagnostic plots don't tell us very much, but we can conclude (top right) that the residuals are very close to Normal (although this is the least important of the assumptions of the linear model); (bottom left) that, after transforming, the variances are almost identical between groups; (bottom right) that there are no particularly influential points (the Cook's distance = 0.5 contour is barely visible on the plot).

Campbell, Harlan. “The Consequences of Checking for Zero-Inflation and Overdispersion in the Analysis of Count Data.” Methods in Ecology and Evolution 12, no. 4 (2021): 665–80. https://doi.org/10.1111/2041-210X.13559.

Campbell, H., and C. B. Dean. “The Consequences of Proportional Hazards Based Model Selection.” Statistics in Medicine 33, no. 6 (2014): 1042–56. https://doi.org/10.1002/sim.6021.

Ruxton, G. D. “The Unequal Variance T-Test Is an Underused Alternative to Student’s t-Test and the Mann-Whitney U Test.” Behavioral Ecology 17, no. 4 (April 2006): 688–90. https://doi.org/10.1093/beheco/ark016.

Improve this answer
$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.