2
$\begingroup$

I have the summary results of a linear regression model with 26 covariates which has been fitted to two independent samples (with different sizes). Now I am planning to use the linear regression beta estimates of a covariate of interest within this model and its standard errors across the two studies to perform an inverse-variance-based meta-analysis of the betas and the p-value of its significance. Since I need to use the t-statistics in this process for tracking deviation of the meta-analysis beta from zero, I wonder what would be the correct degrees of freedom for converting the meta-t statistics to the final meta analysis p-value, as the two samples have different sizes, and therefore DOFs?

I already read this previous question How to perform a meta-analysis of regression coefficients? but seems like this does not answer my confusion with DOF in the meta-analysis stage.

$\endgroup$
11
  • $\begingroup$ Why are you using the t-statistic for testing the statistical significance of the meta-analytic mean beta? Typically, inverse-variance weighted meta-analysis uses the z-distribution. There is some debate on this issue and some argue for using t. Give that this will be a mean of two beta's the DOF is 1. $\endgroup$
    – dbwilson
    Jun 6 '18 at 15:14
  • $\begingroup$ @dbwilson I'm a bit confused. In a simple linear regression model, wouldn't we use the t-statistic for testing significant deviation of a beta estimate from zero? Then how come this may change to z in the meta analysis step? $\endgroup$
    – Sourena
    Jun 7 '18 at 7:22
  • $\begingroup$ If I understand your data, you have two beta's you want to average with meta-analysis. Each has a different standard error (and is based on a different sample size). To compute the average of these betas you are using inverse variance weighted meta-analysis. The significance of the mean beta is assessed with a z-test. In a fixed effects model, its standard error in the inverse of the sum of the weights (which are the inverse of the squared standard errors). $\endgroup$
    – dbwilson
    Jun 7 '18 at 11:43
  • $\begingroup$ What moves forward to the meta-analysis from the simple OLS regression models is the betas and standard errors, not the t-tests and p-values (the latter two have no unique information not contained in the estimate and its standard error). $\endgroup$
    – dbwilson
    Jun 7 '18 at 11:44
  • $\begingroup$ @dbwilson Now I'm starting to get it. Just one last question, if we can use z-transform (and the z-test) to assess deviation of the meta-beta estimate from zero using the inverse variance weighting, how does this approach reduce to the extreme case of only one sample meta-analysis (I know it is weird to do a meta test with only one sample). Maybe I am missing something here? $\endgroup$
    – Sourena
    Jun 7 '18 at 12:03
1
$\begingroup$

If I understand your data, you have two beta's you want to average with meta-analysis. Each has a different standard error (and is based on a different sample size). To compute the average of these betas you are using inverse variance weighted meta-analysis. The significance of the mean beta is assessed with a z-test. In a fixed effects model, its standard error in the inverse of the sum of the weights (which are the inverse of the squared standard errors).

What moves forward to the meta-analysis from the simple OLS regression models is the betas and standard errors, not the t-tests and p-values (the latter two have no unique information not contained in the estimate and its standard error).

The reason that we don't use a t-test to assess the significance of the mean beta is that this beta is a meta-analytic mean and information on its precision comes from the estimates of the precision of the two betas on which it is based. There is an important caveat. This is assuming a fixed effect model. With only two estimates you cannot get any reasonable estimate of the between-study variance for a random effects model.

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.