# Tag Info

21

This response focuses on the second question, but in the process a partial answer to the first question (guidelines for a QA/QC procedure) will emerge. By far the best thing you can do is check data quality at the time entry is attempted. The user checks and reports are labor-intensive and so should be reserved for later in the process, as late as is ...

14

There is actually quite a bit of a debate in the literature whether one should conduct a meta-analysis with the raw correlation coefficients or with the r-to-z transformed values. However, leaving aside this discussion, there are really two reasons why the transformation is applied: Many meta-analytic methods assume that the sampling distribution of the ...

13

(1) As already mentioned by @PeterFlom, one explanation might be related to the "file drawer" problem. (2) @Zen also mentioned the case where the author(s) manipulate(s) the data or the models (e.g. data dredging). (3) However, we do not test hypotheses on a purely random basis. That is, hypotheses are not chosen by chance but we have (more or less strong) ...

12

In my experience doing them, if they haven't been done before, as in you're not providing your own twist on an area, then the right journals don't have bias against them. A meta-analysis won't get in Science but in your field good journals are usually fine with new meta-analyses. The time and cost saved not doing an experiment is often eaten up doing other ...

11

You can calculate/approximate the standard errors via the p-values. First, convert the two-sided p-values into one-sided p-values by dividing them by 2. So you get $p = .0115$ and $p = .007$. Then convert these p-values to the corresponding z-values. For $p = .0115$, this is $z = -2.273$ and for $p = .007$, this is $z = -2.457$ (they are negative, since the ...

10

Great Question! In the scientific context there are various kinds of problematic reporting and problematic behaviour: Fraud: I'd define fraud as a deliberate intention on the part of the author or analyst to misrepresent the results and where the misrepresentation is of a sufficiently grave nature. The main example being complete fabrication of raw data ...

10

Q: Can I still make a funnel plot with effect size on the horizontal axon and total sample size n (n=n1+n2) on the vertical axis? A: Yes Q: How should such a funnel plot be interpreted? A: It is still a funnel plot. However, funnel plots should be interpreted with caution. For example, if you have only 5-10 effect sizes, a funnel plot is useless. ...

9

I am not an expert, but I think it is common sense to not merge two studies if they have strongly opposite outcomes. For the sake of the example, say they measure a variable $X$ (glucose in the blood etc.) in identical conditions. Since the outcome is different, you can imagine that there is "something" unknown, so that one of the team measures actually $X ... 9 One argument that is missing so far is the flexibility of data analysis known as researchers degrees of freedom. In every analysis there are many decisions to be made, where to set the outlier criterion, how to transform the data, and ... This was recently raised in an influential article by Simmons, Nelson and Simonsohn: Simmons, J. P., Nelson, L. D., ... 9 Introduction to Meta-Analysis by Borenstein, Hedges, Higgins and Rothstein provides a detailed discussion of the pros and cons of meta-analysis. See for example the chapter "Criticism of meta-analysis" where the authors respond to various criticisms of meta-analysis. I note the section headings for that chapter and then make a few observations from memory ... 9 Answering this (good) question responsibly probably requires addressing meta-analysis topics beyond conventional meta-regression. I've encountered this issue in consulting clients' meta-analyses but haven't yet found or developed a satisfactory solution, so this answer isn't definitive. Below I mention five relevant ideas with selected reference citations. ... 8 Create a proper data.frame: df <- structure(list(study = structure(c(1L, 5L, 3L, 4L, 2L), .Label = c("Foo2000", "Pete2008", "Pric2005", "Rota2008", "Sun2003"), class = "factor"), mean1 = c(0.78, 0.74, 0.75, 0.62, 0.68), sd1 = c(0.05, 0.08, 0.12, 0.05, 0.03), n1 = c(20L, 30L, 20L, 24L, 10L), mean2 = c(0.82, 0.72, 0.74, 0.66, 0.68), sd2 = ... 8 As @Alexander already mentioned, you are looking for an approach that is called "individual participant/person/patient data meta-analysis" (IPD meta-analysis). He also refered to an article by Richard Riley, who has published a lot in this field. Please find below a collection of articles that I used for our advanced meta-analysis class: Cooper, H., & ... 7 You could start with David B Wilson's website on "meta-analysis stuff". He offers spss, stata, and sas macros for performing meta-analytic analyses (including meta-regression; metareg.sps) + PPT slides (analysis.ppt, interpretation.ppt). Another presentation I really like(d) was given by Marsh et al. „Meta-Analysis: Session 3.3 & 3.4: Teacher Expectancy ... 7 Your LSD equation looks fine. If you want to get back to variance and you have a summary statistic that says something about variability or significance of an effect then you can almost always get back to variance—-you just need to know the formula. For example, in your equation for LSD you want to solve for MSE, MSE = (LSD/t_)^2 / 2 * b 7 You can do that with the funnel() function from the metafor package. Here is an example: library(metafor) data(dat.bcg) res <- rma(ai=tpos, bi=tneg, ci=cpos, di=cneg, data=dat.bcg, measure="RR", method="REML") ablat.scaled <- with(dat.bcg, (ablat - min(ablat))/(max(ablat) - min(ablat))) ablat.scaled <- ablat.scaled * 2 + 0.5 funnel(res, ... 7 I thank @Bernd for pointing me in the right direction. Here are some notes on the references he mentioned in his answer, as well as some of the references mentioned in these articles. Sutton et al (2008) Sutton et al use within a health context the terms individual patient data versus aggregate data. They note that analysis of individual patient data is ... 7 I think it is a combination of everything that has already been said. This is very interesting data and I have not thought of looking at p-value distributions like this before. If the null hypothesis is true the p-value would be uniform. But of course with published results we would not see uniformity for many reasons. We do the study because we expect ... 7 Built on Adam's answers, I have a few elaborations. First and most important, it is not easy to conceptualize substantive theories on how and why one effect size predicts another effect size. A multivariate meta-analysis is usually sufficient to explain the association among the effect sizes. If you are interested in hypothesizing directions among the effect ... 6 If you have all this data, I think the best answer is to actually fit a single large model, using Hierarchical Modeling rather than do it in two steps (generating a prior then fitting a model). This is basically the answer I gave to this question. I explain this a little bit more there. In a hierarchical model you model each of the parameters you are ... 6 I can only agree with John. Furthermore, perhaps this paper by David Saville helps you with some formula to recalculate variability measures from LSDs et al.: Saville D.J. (2003). Basic statistics and the inconsistency of multiple comparison procedures. Canadian Journal of Experimental Psychology, 57, 167–175 UPDATE: If you are looking for more formulas to ... 6 Yes, it is possible, but whether it is appropriate depends on the intent of your analysis. Meta-analysis is a method of combining information from different sources, so it is technically possible to do a meta-analysis of only two studies - even of multiple results within a single paper. The key concern is not if you can do this, but that the method is ... 6 The criterion for a 'trivial' effect size (odds ratio in your example) should be decided based on the size of effect that would be considered 'trivial' in the particular scenario, rather than on statistical grounds. If you're looking at an intervention that may be given to a considerable segment of the population with few side-effects and may prevent early ... 6 Your question indicates, to me, you're not yet ready to embark on the data abstraction portion of your meta-analysis. Your question needs refining, and you need to decide exactly what you're interested in asking. In your examples above, you appear to be interested in the main reported effects of the RCTs, which are found in the following places: "Among the ... 6 This web page ("Forest plots : Introduction and explanation") explains how to draw simple forest plots in MS-Excel. Is your friend conducting a meta-analysis? In that case s/he might be interested in the following two pages: MIX 2.0 - Meta-Analysis made easy MetaEasy Excel add-in 5 In order to conduct Egger's regression test you will also need the standard errors ($SE_i$) of your effect sizes ($ES_i$). Then generate the so called standard normal deviate (SND) which is defined as effect size divided by its standard error ($ES_i / SE_i$). Next, generate the precision which is$\frac{1}{SE_i}$. The regression model is:$SND = a + b \cdot ...

5

Your derivation is perfectly fine for regular (non-repeated meaures) ANOVA. For repeated measures ANOVA the F-statistics does not always equal $MS_g/MS_e$. Assuming weeks is the repeated measure, this formula is correct for the weeks and weeks*treatmeant terms (note that they will give the same MSE), but not for the treatment term. There is another term that ...

5

In most meta-analysis of odds ratios, the standard errors $se_i$ are based on the log odds ratios $log(OR_i)$. So, do you happen to know how your $se_i$ have been estimated (and what metric they reflect? $OR$ or $log(OR)$)? Given that the $se_i$ are based on $log(OR_i)$, then the pooled standard error (under a fixed effect model) can be easily computed. ...

5

Do you mean interpreting this? Here are a couple of other descriptions: 1.Interpreting Funnel Plots 2."Funnel plots in Meta Analysis" by Sterne & Harbord 3."Funnel plots for detecting bias in meta-analysis: guidelines on choice of axis" by Sterne & Egger But the wikipedia link is probably the best place to start. (It also contains a few other ...

Only top voted, non community-wiki answers of a minimum length are eligible