Without multi-level modeling, how to handle within-study replication in a meta-analysis, where the study is the unit of replication? Description of the study:
I have observed a common error among meta-analyses, with regard to handling of within-study replication. It is not clear to me if the error invalidates the studies when assumptions are stated. However, as I understand it, these assumptions violate a basic premise of statistics.
As an example, a study tests the effects of chemical $X$ on response $Y$.
The analysis is performed on the log response ratio: the ratio of treatment $Y_{+X}$ (in the presence of $X$) to control $Y_{0}$ (no $X$):
$$R = \ln(\frac{Y_{+X}}{Y_{0}})$$
Some of the studies included in the meta-analysis contain multiple treatments, for example different levels or chemical forms of $X$. For each treatment, there is a different value of $R$, although $R$ always uses the same value of $Y_0$. 
The methods state:

responses to different treatments (levels and forms of $X$) within a single study were considered independent observations.

Questions:


*

*Isn't this pseudoreplication? 

*Is it inappropriate even if the violation of independence is stated in the methods? 

*What would be an easy way (e.g. within the ability of a simple meta-analysis software package) to handle within study replication?


Initial thoughts:


*

*Summarize results of each study, e.g. by taking the average response

*Select only one treatment from each study based on a-priori criteria (e.g. highest dose, first measurement)?


Are there any other solutions?
 A: Yes, it is a problem because there is sampling dependence in the
responses which needed to be accounted for (although sometimes the effect might be negligible and we do violate assumption all the time when we perform statistical analyses). There are methods to deal with this, one approach is to include the covariances between related experiments (off-diagonal blocks) in the error variance-covariance matrix (see e.g. Hedges et al., 2010). Fortunately with log ratios this is rather easy. You can get approximated covariances between experiments because the variance (var) of log R is (if Yx and Y0 are independent groups): log Yx - log Y0, to follow the notation in the question, Yx referring to the experimental group and Y0 the control group. The covariance (cov) between two values (e.g. treatment 1 och treatment 2) for log R is cov(loge Yx_1 - log Y0, log Yx_2 - log Y0), which equals var(log Y0), and is calculated as the SD_Y0/(n_Y0 * Y0), where SD_Y0 is the standard deviation of Y0, n_Y0 is the sample size in the control treatment, and Y0 is the value in the control treatment. Now we can plug-in the whole variance-covariance matrix into our model instead of only using the variances (ei) which is the classic way to perform a meta-analysis. An example of this can be found in Limpens et al. 2011 using the metahdep package in R (on bioconductor), or Stevens and Taylor 2009 for Hedge´s D.
If you want to keep it very simple, I would be tempted to ignore the problem and try to evaluate the effect of sampling dependence (e.g. how many treatments are there within studies? how do the results change if I only use one treatment? etc) .
A: Yes, this is a problem.
Yes, it is inappropriate even though at least it is transparent about what it is doing (it gets points for transparency, but still is not satisfactory).
I doubt there is an "easy way" to fix this.  I don't know much about the approaches taken to meta analysis but if there is specific meta-analysis software and research like this is produced using it and gets published, this may well be the common approach.  Either of your proposed responses loses some granularity of information from each study (ie the opposite problem of what the publishers have done).
The obvious solution is a mixed-effects (ie multilevel) model with study as a random factor.  I would suggest using a specialist statistical package for this if meta-analysis software cannot do it.  You could still use the meta-analysis software for data storage and processing, and just export data to R, Stata or SAS for the analysis.
