Pick a card, any card: understanding the frequentist interpretation of a confidence interval I've been trying to wrap my head around the oft-quoted misconceptions surrounding the frequentist interpretation of a confidence interval. There are many questions on Cross Validated and many excellent and interesting answers (such as Clarification on interpreting confidence intervals?). It seems to be the general consensus that a 95% confidence interval, for example, should be interpreted in terms of repeating an experiment multiple times and, under such circumstances, the calculated interval will contain the true parameter value 95% of the time. However, it is also clear that most contributors to this site agree that this should not be interpreted as there being a 95% probability that a single confidence interval calculated from a random sample will contain the true (fixed but unknown) parameter value. And the reason appears to be that the frequentist interpretation of a confidence interval relies on long-run frequencies (again see: Clarification on interpreting confidence intervals?). The true parameter value is either in the interval or not and, therefore, probability does not come into it.
Perhaps the reason that I have such a hard time understanding this issue is that it seems to fly in the face of the very earliest statistical concepts that I – and presumably many others – was taught, namely the probability associated with games of chance. If I select a card at random from a well-shuffled standard deck and place it face-down on the table, the probability that the card is a club is 13/52 = 0.25. But, using the same reasoning as that applied to confidence intervals, should I avoid thinking in these terms? The card either is or is not a club and there are no long-run frequencies to consider. So is it legitimate – using frequentist philosophy – to say that the randomly chosen card that I have selected from a deck and placed face down on the table has a 25% probability of being a club?
 A: A Frequentist might say: If you have a concrete 95% confidence interval, calculated from an observed dataset, then the parameter either is or isn't in that interval. But a 95% confidence interval procedure [draw a random sample from a well-defined population, perform certain calculations on that dataset, and report the resulting interval] is designed to cover the parameter 95% of the time.
I can never be certain whether any specific interval covers the parameter, so I've just decided to be satisfied when I know the interval came from a procedure with 95% coverage. But I wouldn't trust an interval if I knew it came from a procedure with unknown coverage [such as if the data arose from nonrandom sampling, or if there were bugs in the CI calculation software].
Similarly, in your card example: The card either is or isn't a club. But the procedure you used to choose the card [draw one card at random from a well-shuffled deck] is designed to choose a club 25% of the time.
I can never be certain whether any newly-dealt, face-down card is a club, but in many card games I'm OK with being dealt a card using a process that has 25% chance of clubs. But I wouldn't want to play if I knew the card was dealt using a process with unknown chance of clubs [such as if the dealer cheated and didn't shuffle].

So is it legitimate – using frequentist philosophy – to say that the randomly chosen card that I have selected from a deck and placed face down on the table has a 25% probability of being a club?

Ultimately, it depends on what you are trying to communicate. Often it's fine to say "There's a 25% chance that this face-down card is a club" and even a Frequentist will understand from context that it's just meant as convenient shorthand, not a misunderstanding of probability theory. But if you're working through a technical proof and you need to be carefully explicit, a Frequentist might prefer to say "This face-down card was selected using a process that has 25% chance of producing a club."
A: I agree with the answer by @civilstat. I would just like to add an example which helps me to understand the difference.
Consider the following procedure: you take a random sample from a population and use the sample mean to construct a confidence interval for the true mean. The confidence interval is constructed using the following procedure: randomly choose $(-\infty, \infty)$ with probability 0.95 and the empty interval with probability 0.05. This satisfies the definition of a confidence interval; it's just not a very useful one.
Now, there's a 95% chance that the interval constructed using your procedure contains the true mean. So, if I was to do the experiment and construct my interval without showing it to you, it would make sense to say that my interval has a 95% chance of containing the true mean. This is analogous to drawing your card and putting it face down on the table.
But if I was to show you my interval and it's empty, it would be a bit silly to say that it has a 95% chance of containing a true mean, when it clearly doesn't. This would be like turning over your card, noticing that it's a heart, and still saying that it has a 25% chance of being a club.
The problem is in the name: we call it a "confidence interval" but it's not an interval; it's a procedure for constructing an interval. The analogy in the case of playing cards would be like saying "playing card" instead of "playing card drawn at random". Saying that a confidence interval has a 95% chance of containing the true mean is like saying "a playing card has a 25% chance of being a club". Which sounds reasonable. But the statement "a playing card has a 25% chance of being a club" can't be taken literally. Otherwise the King of Hearts has a 25% chance of being a club.
A: You know the exact demographics of the population in your card deck scenario. Frequentist statistics, statistics by definition, is used when we don't know the population, but rather we have a sample. That would be like taking X number of cards at random and estimating the probability (proportion) of clubs in the population. 
