Let's say I'm looking at a forest plot from a meta-analysis. I notice that most of the width of the confidence intervals are fairly consistent and the point estimates are all on one side (showing a benefit). Is it possible to use these trends to inform us on what future trials with the same research question will be? (I don't know how else to ask this)
5
-
1$\begingroup$ Questions should stand alone even if their links were to disappear. Please edit your question so there's sufficient context in your question to answer it (you may need to quote some section of the material). A link that supplements the question is good, but it should still make sense if the link were no longer there. (Briefly, the abstract doesn't say anything remotely controversial that I see.) $\endgroup$– Glen_bCommented Aug 30, 2021 at 0:59
-
1$\begingroup$ Geoff Cumming has put a lot of work into designing simulations that can be run to explore issues exactly like this. Perhaps you should have a play with them and see whether you can tune your intuition: thenewstatistics.com/itns/esci $\endgroup$– Michael LewCommented Aug 30, 2021 at 1:47
-
$\begingroup$ @Glen_b Thank you Glen. I corrected my error and hopefully I've made it more clear what I'm asking. $\endgroup$– Dylan ACommented Aug 30, 2021 at 3:35
-
$\begingroup$ @MichaelLew Thank you! $\endgroup$– Dylan ACommented Aug 30, 2021 at 3:35
-
1$\begingroup$ @dylan the question is rather open ended/vague. I think you need more context relating to the specific claims there that you're asking about $\endgroup$– Glen_bCommented Aug 30, 2021 at 3:59
Add a comment
|
1 Answer
$\begingroup$
$\endgroup$
Confidence intervals always rely on the data that was at hand at the time of analysis and are not necessarily applicable to future studies.
If nothing changes in the world and you make the same measurements as the original study made, your mean value is likely to lie within the confidence bounds. However, if the situation changed in ways that are factored into the model, your data will deviate from the original study.
For example: If I count people at the beach every day in August, I get a mean and a CI and any other person counting people in August will likely come to a mean that lies within the confidence interval. That is not to say, however, that someone counting in January will get a mean within these bounds.
But if I counted people on the beach every month and present a model with estimated means for each month, anyone counting in any month may come to a mean within the bounds for this month.
If, however, I collected this data from 2000 to 2018 and someone counts people at the beach in April 2020, the mean will not lie within confidence bounds of the month April as I never counted during a pandemic and this situation is novel.
So, whether or no the confidence interval in a study informs you about the expected outcome of your own study depends on the context in which the original study was undertaken, whether th same conditions apply to your case and whether changed parameters are factored into the original study. Whether this applies to your particular case, I do not know as the question was quite vague.
