# Confidence interval overlap - judging the similarity of estimates

I have been tinkering with an idea today, but I have only taken one semester of statistics, so please bear that in mind. Thanks. First up, the question:

How do you judge multiple estimates (with standard errors included)? I don't mean estimating their mean or distribution, rather some representation of similarity. Something like R^2 (or a different goodness-of-fit measure) in linear regression models.

I have heard of meta-analysis, but that, to my knowledge, deals with the estimation, precision and requires a bit more data. I does not, AFAIK, say much about the similarity. Maybe the size of the CI of the estimate does reflect similarity in a way.

When we tested for similarity in school, we did not consider the errors, only the estimates (chi-square test, Fisher test).

My thoughts:

When you have two estimates and their standard errors, you are interested in their confidence intervals. While I know it's dangerous to judge the statistical similarity only based on these intervals instead of hypotheses tests, but what about checking the size of the overlap? Getting a ratio of something like $$ratio_{1,2} = \frac{l(overlap_{1,2})}{l(CI_1) + l(CI_2)},$$ where $l$ only returns the length of an interval. The closer to one, the better (that of course does not have to hold). I started thinking about it, because sometimes you have $n$ estimates for the same phenomenon and you would like to know the level of their similarity. For example a ratio

$$\frac{\sum_{i,j,i<j} ratio_{i,j}}{n(n-1)/2},$$ would be an indicator of pairwise similarity.

Just a thought. Just a brain dump from when I was walking my dog.

I Googled a bit and found two papers on the topic, but they are a bit too technical for me, they also deal with problems kind of different.

I am sorry if this is some kind of trivial topic in statistics. Or if the description of the question is not clear.

Thanks!

• Could you please clarify what you mean by estimates. Do you mean, for e.g., that you want to test the overlap between height and weight for boys? Or, do you want to test the overlap between boy height and girl height? – Michelle Jan 26 '12 at 9:45
• I mean estimates for the same phenomenon. For example estimation of, say, price elasticity of [...]. There are multiple studies with varying data, methodologies (but all yielding estimates with st. errors, say using linear regressions), ... so their estimates are most likely a bit different, but I would like to know how different, or how similar. // So in your example it's the former. – Ondrej Jan 26 '12 at 13:00