Real World Consequences of Misinterpreting Confidence Intervals? I understand that pre-data collection we can be 95% confident that the interval we're about to calculate will contain the true population parameter θ.
This doesn't mean that once we've calculated the C.I. that there's a 95% chance that θ is in the C.I.
θ is either contained within the interval or it isn't. As now both θ and the C.I. are fixed quantities
However, if we wished to calculate an interval with a 95% probability that that interval contained θ we could use a Bayesian credible interval.
I've seen this discussion in many places. These discussions also tend to include how often the misinterpretation of C.I. appear in the research literature and how few researchers understand the correct interpretation when polled, or quizzed on the topic.
What I've failed to find is the real-world consequences of interpreting a C.I
as if it were a credible interval.
Can someone please point me in the direction of real-world consequences? Preferably a case-study, but even an example would be greatly appreciated.
 A: Where the confidence and credible intervals have similar bounds it is hard to see that interpreting one as if it were the other is an important problem. 
I have not come across a real-world circumstance where the distinction between a fixed unknown parameter value and a parameter value that is a random variable is problematical of itself. However, it is important to realise that credible intervals can be shaped by an informative prior and such an interval will be different from the frequentist confidence interval derived for the same data. In that circumstance it would be more problematical to assume that the confidence interval reflected the (posterior) probability of the parameter values of interest.
A: Firstly, there's artificial examples. E.g. a 95% CI that contains the whole parameter space 95% of the time and the is just the interval from 11.23234576 to 11.2323457 5% of the time is obviously a valid 95% CI (i.e. contains the true value at least 95% of the time), but also completely useless. Nobody uses that kind of thing in practice, so let's ignore these types of examples.
Secondly, there's the situation where you use something like sample mean $\pm$ 1.96 $\times$ standard error. I.e. you are effectively using flat priors. If you truly knew absolutely nothing about the problem at hand, I guess that might be a reasonable prior belief and okay. In practice, we are hardly ever in that situation. We often have some idea of the variability of the measurement, as well as location (e.g. we have some idea by how much someone's blood pressure can change without treatment, we have some idea how much of an effect a new blood pressure drug could possibly have etc.). If one ignores that prior information, then extreme estimates and confidence intervals created using flat priors are much more likely to not contain the true value than the ones that are more consistent with the prior information.
In practice, I believe the issue is usually from the second case.
This is a real issue e.g. for a drug company that has a series of drugs coming out of pre-clinical research and there's not that much to distinguish them (all are considered sort of promising and showed some promising results in in-vitro and animal experiments). Now, these drugs are each being tested in small proof-of-concept studies (especially so, if these are powered for e.g. 80% power at the 10% one-sided significance level for very optimistic assumed effect sizes). If about 30 to 50% of the new drugs being tried really have a meaningfully large effect (with some obvious bounds on that), then confidence intervals containing very large effect sizes are more likely to lie above the true effect than those that are less "optimistic". If you then need to make decisions (proceed or not) and for determining the size of the next studies (and likelihood of success), using the confidence intervals as if they were 95% credible intervals is likely being much too optimistic. That's why in practice people will take much more Bayesian approaches (see e.g. what one company does and another).
A: Much research is done in many fields. In many journals and fields statistical significance is seen as a requirement for a "discovery" worth publishing. This means that there is publication bias; significant studies (i.e. studies where the confidence interval does not include "no effect", often formalised as parameter zero) are published and insignificant ones disappear, and people look at many things until they find something significant.
The consequence of this is that published significant effects are actually biased high; in other words, confidence intervals are too far on one side, away from "no effect".
Using credible intervals should force the researcher to think about prior information and plausibility. (In fact this often doesn't happen, supposedly "informationless" priors are used, and the situation may not be better than with confidence intervals.) Andrew Gelman argues on his blog that in many social science studies priors should be chosen so that small and zero effects are more and large effects are less likely, so that credible intervals have a better chance to include zero and small effects, mitigating the publication bias problem.
There are various references on the blog to studies that find significant and in fact implausibly large effects using frequentist inference, e.g., that more attractive parents tend to have daughters rather than sons, or that voting behaviour of women depends on how long after ovulation the election takes place. It's somewhat hard to find a specific one using the blog search system but there are many of them. One related posting is
this.
I admit though that this issue is only somewhat loosely related to the question, as the problem with confidence intervals here is not in the first place that they are wrongly interpreted as credible intervals (and in fact credible intervals may mess things up as well with an unsuitable prior). There is some relation though. The bare fact that the confidence interval has large effect sizes in it shouldn't lead us to believe that these are plausible or true with high probability. They are mathematically correct, but the confidence level is a performance characteristic rather than a measure of plausibility/probability of parameters, and the performance characteristic has limited value or even requires adjustment in case many confidence intervals are in fact run, or they are run conditionally on other diagnoses performed on the same data.
Bayesian analyses grant epistemic interpretation, i.e., probabilities assigned to parameters regard our knowledge/expectations of the characteristics of the underlying process rather than the performance of the method. This will however not necessarily solve the problem. In particular, publication and selection bias can still bite if results based on priors are favoured that lead to headline-grabbing claims or if priors are chosen dependent on the data. Furthermore all Bayesian results are of course conditional on prior and model, and can only do better if these are chosen taken information appropriately into account. Often "informationless priors" are used that simply reproduce a problematic frequentist analysis.
I should also mention that frequentists argue that we actually should be interested in performance characteristics, and that it is a bug rather than a feature of Bayesian analyses that they don't bother with this.
A: A 95% credible interval has a 95% posterior probability of containing the parameter. It seems very unlikely that anyone would interpret the 95% confidence level as a 95% posterior probability, and hence it is unlikely that anyone would interpret a CI as if it were a credible interval, contrary to the implications in your question. Rather, the common misinterpretation of CIs by non-statisticians revolves around not understanding the distinction between pre-sample and post-sample, and hence confusing pre-sample 95% probability with post-sample 95% confidence. The latter probability is not a posterior probability.
