6
$\begingroup$

I am trying to manage a meta-regression in SPSS17 using the effect size as the dependent variable. I want to explore if my independent variables affects the effect size. Some small practical questions:

  1. What is the minimum number of studies necessary for a meta-regression?

    • Some people suggest at least 10 studies are required. Why not 20 or 5 studies?
    • Is the total sample size an important consideration?
    • Why would 10 studies with 200 patients be enough, but 5 studies with 400 patients not be enough?
  2. Can I enter all three regressors at once and report the global model, or do I have to enter one regressor at a time and report 3 models each one separately?

    • How does the correlation between the independent variables affects this choice?
    • How does the number of the studies affect the number of independent variables that I should enter simultaneously?
  3. Does the independent variable have to be a scale variable?

    • The dependent variable (=effect size) is, of course, scale. The independent variable must be also scale, or could be ordinal or nominal?
  4. How can I weight my effect size for sample size?

  5. What is the preferable level of significance?

    • Is p<0.05 still acceptable for clinical research in such an analysis?
$\endgroup$
4
  • 1
    $\begingroup$ Your topic is interesting, but you ask so many questions about different aspects of modeling and meta-analysis, and the questions are so much of a judgment-oriented rather than a how-to nature, that I'm afraid you're not going to get them answered satisfactorily in this venue. If you could narrow down.... $\endgroup$ – rolando2 Feb 24 '11 at 0:41
  • $\begingroup$ @rolando2 I agree, but the topic is hot. Also there is no meta-regression tag available. I hope that someone would spend time to add a paragraph for reference! $\endgroup$ – Staty Despair Feb 24 '11 at 0:55
  • 1
    $\begingroup$ I modified your list of tags, hope you agree. Before I start answering your questions, I'd like to know how many data points you have. $\endgroup$ – Bernd Weiss Feb 24 '11 at 1:18
  • $\begingroup$ @Bernd Twenty included studies, three independent variables, let's say 60 data points more or less, for the meta-regression. $\endgroup$ – Staty Despair Feb 24 '11 at 1:47
6
$\begingroup$

Don't use the built-in routines of SPSS to conduct a meta-regression (wrong standard errors; does not give you correct model indices; no heterogeneity statistics). Have a look at David Wilson's SPSS "macros for performing meta-analytic analyses". One of these macros is called MetaReg which can perform fixed-effect or mixed-effects meta-regression. I would always use Stata or R. By the way, user Wolfgang is the author of an R package called metafor. This is an excellent piece of software to conduct meta-regression.

As a general (non-technical) intro to meta-regression, I can recommend Thompson/Higgins (2002) "How should meta-regression analyses be undertaken and interpreted?".

Now to your question:

Q1: What is the minimum number of studies necessary for a meta-regression? Some people suggest at least 10 studies are required. Why not 20 or 5 studies?

The answer can be found in Borenstein et al (2009: 188):

"As is true in primary studies, where we need an appropriately large ratio of subjects to covariates in order for the analysis be to meaningful, in meta-analysis we need an appropriately large ratio of studies to covariates. Therefore, the use of metaregression, especially with multiple covariates, is not a recommended option when the number of studies is small. In primary studies some have recommended a ratio of at least ten subjects for each covariate, which would correspond to ten studies for each covariate in meta-regression. In fact, though, there are no hard and fast rules in either case."

Q2: Is the total sample size an important consideration?

What is total sample size? The number of studies? Yes, it is important. Or the number of individuals? No, it is not (or less) important.

Q3: Why would 10 studies with 200 patients be enough, but 5 studies with 400 patients not be enough?

It is just a(n ordinary) regression. You wouldn't run a regression with 5 data points, would you? In your comment, you state that you have 20 studies which is enough to run a meta-regression.

Q4: Can I enter all three regressors at once and report the global model, or do I have to enter one regressor at a time and report 3 models each one separately?

It is just a regression. I would start with three simple bivariate models then build more complex models (be aware of multicollinearity, see below).

Q5: How does the correlation between the independent variables affects this choice?

A high correlation between your predictor variables will have a (negative) impact on your results. You should avoid that. Please consult a textbook for the problem of multicollinearity.

Q6: How does the number of the studies affect the number of independent variables that I should enter simultaneously?

See the Borenstein et al citation.

Q7: Does the independent variable have to be a scale variable? [...] The independent variable must be also scale, or could be ordinal or nominal?

What is a "scale variable"? Do you mean a continuous/metric variable? You predictor variables can have any level of measurement. However, if you have a categorical (nominal) predictor variable, you will have to deal with dummy variables (see Multiple Regression with Categorical Variables).

Q8: How can I weight my effect size for sample size?

As far as I know, all meta-regression approaches expect the weights to be the inverse study variance, i.e. $\frac{1}{v_i}=\frac{1}{SE_i^2}$. Again, you will need standard errors :-)

Q9: What is the preferable level of significance? Is p<0.05 still acceptable for clinical research in such an analysis?

I cannot answer your question. That really depends on your research question. In my (non-clinical) research I am happy with p < 0.10.

$\endgroup$
4
  • $\begingroup$ Thank you for your excellent answer and references. I installed R to try metafor. More difficult than I was expected. There are many tutorials available but none for non-statisticians :-) $\endgroup$ – Staty Despair Feb 27 '11 at 2:41
  • 1
    $\begingroup$ @ Staty Despair: You might be interested in Yet another short introduction to R with some emphasis on meta-analysis. $\endgroup$ – Bernd Weiss Feb 27 '11 at 11:44
  • $\begingroup$ great job, undoubtendly! I don't know why I didn't find this text last days I searched hundrends of R related pages. Nevertheless, i still can't manage a simple meta-analysis using R, ...despite the fact that I already know the results from Cochrane RevMan. The missing piece of the puzzle, is a simple example of a real-life meta-analysis problem, solved by keeping the R syntax as simple/realistic as possible. "One word of truth outweighs the whole world" (Alexander Solzhenitsyn, 1970, Nobel lecture). $\endgroup$ – Staty Despair Mar 2 '11 at 1:16
  • 2
    $\begingroup$ @BerndWeiss Just a note that as of now, the repository for that link no longer exists. I've done a quick search for it, but haven't been able to find it. In the meantime, there's another tutorial here for meta-analysis in R. $\endgroup$ – sebastian-c May 27 '14 at 17:03
2
$\begingroup$

These are some wonderful responses to your initial questions, and the reference guide is particularly helpful.

If you're looking for a relatively simple package to do meta-regression, may I recommend Borenstein 's software package Comprehensive Meta Analysis. It is limited to meta-regression of a single predictor but this can be scale/continuous or it can be categorical. The output and graphics produced are easy to follow, and unless you're using multiple predictors it will suffice. Given that you're talking about a small study size (eg 10) you probably wouldn't be able to detect multiple predictors anyway.

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.