3
$\begingroup$

I have data where I estimate regression coefficients/slopes for two groups (e.g. sex-specific regression coefficients) for a number of single effect regression models. I have used Bayesian methods which allow me to test, individually for each model, whether the regression coefficients of each group are significantly different (and they are generally not significant).

However, there seems to be a trend, that one group has consistently lower coefficients (nearer to zero) than the other.

Could I then use (/would it be appropriate to use) a paired t-test to see if the (absolute) value of the regression coefficients for one group is consistently lower than the other to test whether one group is more sensitive to explanatory variables?

E.g. (in R)

> a = data.frame("Coefficient" = c(1:10), "GroupA" = abs(rnorm(10,1,1)), "GroupB" = abs(rnorm(10,4,1)))
> a
   Coefficient    GroupA   GroupB
1            1 1.2359143 4.528682
2            2 0.1260180 5.703323
3            3 1.1529601 5.998172
4            4 2.3689296 4.343029
5            5 1.8734228 4.245404
6            6 1.1287780 4.699337
7            7 1.8684325 3.829195
8            8 0.2723389 4.646488
9            9 3.1309934 4.158523
10          10 3.3550409 5.042786
> t.test(a[,2],a[,3],paired=TRUE)

        Paired t-test

data:  a[, 2] and a[, 3]
t = -6.4382, df = 9, p-value = 0.0001198
alternative hypothesis: true difference in means is not equal to 0
95 percent confidence interval:
 -4.146265 -1.990157
sample estimates:
mean of the differences 
              -3.068211
Improve this question
$\endgroup$
18
  • $\begingroup$ Are these coefficients from different models run on the same dataset, different subsets of the same dataset or different datasets entirely ? $\endgroup$
    – Robert Long
    Jul 6, 2016 at 8:50
  • $\begingroup$ all subsets of the same data (for the response variable and groups), different explanatory variables used in each model. It's data on a trait and regression models test sex-specific responses to various environmental variables. I want to test whether one sex is less sensitive to environment generally (visual suggests that one is consistently less affected than the other, and for some environmental variables this is significant) $\endgroup$
    – rg255
    Jul 6, 2016 at 8:57
  • $\begingroup$ So you are running different models each on a different subset of a larger dataset. The models have one predictor in common that you want to compare across the different models ? $\endgroup$
    – Robert Long
    Jul 6, 2016 at 11:11
  • $\begingroup$ All models have the same response variable, and a common fixed effect of sex (male or female), but vary in the other fixed effect $\endgroup$
    – rg255
    Jul 6, 2016 at 11:50
  • $\begingroup$ IIUC, in the data you have posted, each row refers to a model (run on a different subset of the data), and Group A refers to one sex and Group B to the other sex ? $\endgroup$
    – Robert Long
    Jul 7, 2016 at 7:43

1 Answer 1

Reset to default
2
$\begingroup$

I don't think a paired t-test would be appropriate, since it takes no account of the uncertainty in the individual estimates.

I would suggest:

  1. Run the models on the full dataset in order to achieve greater power.
  2. Conduct a meta-analysis of the results you have on the subsets.
Improve this answer
$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge that you have read and understand our privacy policy and code of conduct.

Not the answer you're looking for? Browse other questions tagged or ask your own question.