Who are frequentists?

Question

We already had a thread asking who are Bayesians and one asking if frequentists are Bayesians, but there was no thread asking directly who are frequentists? This is a question that was asked by @whuber as a comment to this thread and it begs to be answered. Do they exist (are there any self-identified frequentists)? Maybe they were just made-up by Bayesians who needed a scapegoat to blame when criticizing the mainstream statistics?

Meta-comment to the answers that were already given: As contrast, Bayesian statistics are not only defined in terms of using Bayes theorem (non-Bayesians also use it), nor about using subjectivist interpretation of probability (you wouldn't call any layperson saying things like "I bet the chance is less than 50:50!" a Bayesian) - so can we define frequentism only in terms of adopted interpretation of probability? Moreover, statistics $\ne$ applied probability, so should definition of frequentism be focused solely on the interpretation of probability?

Answer 1

Some existing answers talk about statistical inference and some about interpretation of probability, and none clearly makes the distinction. The main purpose of this answer is to make this distinction.

---

The word "frequentism" (and "frequentist") can refer to TWO DIFFERENT THINGS:

1. One is the question about what is the definition or the interpretation of "probability". There are multiple interpretations, "frequentist interpretation" being one of them. Frequentists would be the people adhering to this interpretation.

2. Another is statistical inference about model parameters based on observed data. There is a Bayesian and a frequentist approaches to statistical inference, and frequentists would be the people preferring to use the frequentist approach.

Now comes a speculation: I think there are almost no frequentists of the first kind (P-frequentists), but there are lots of frequentists of the second kind (S-frequentists).

---

Frequentist interpretation of probability

The question of what is probability is a subject of intense ongoing debate with 100+ years of history. It belongs to philosophy. I refer anybody not familiar with this debate to the Interpretations of Probability article in the Stanford Encyclopedia of Philosophy which contains a section on frequentist interpretation(s). Another very readable account that I happen to know of, is this paper: Appleby, 2004, Probability is single-case or nothing -- which is written in the context of foundations of quantum mechanics, but contains sections focusing on what probability is.

Appleby writes:

Frequentism is the position that a probability statement is equivalent to a frequency statement about some suitably chosen ensemble. For instance, according to von Mises [21, 22] the statement “the probability of this coin coming up heads is 0.5” is equivalent to the statement “in an infinite sequence of tosses this coin will come up heads with limiting relative frequency 0.5”.

This might seem reasonable, but there are so many philosophical problems with this definition that one hardly knows where to start. What is the probability that it will rain tomorrow? Meaningless question, because how would we have an infinite sequence of trials. What is the probability of the coin in my pocket coming up heads? A relative frequency of heads in an infinite sequence of tosses, you say? But the coin will wear off and the Sun will go supernova before the infinite sequence can be finished. So we should be talking about a hypothetical infinite sequence. This brings one to the discussion of reference classes etc. etc. In philosophy one does not get away so easily. And by the way, why should the limit exist at all?

Furthermore, what if my coin were to come up heads 50% of the time during the first billion of years but then would start coming up heads only 25% of the time (thought experiment from Appleby)? This means that $P(\mathrm{Heads})=1/4$ by definition. But we will always be observing $\mathrm{Frequency}(\mathrm{Heads})\approx 1/2$ during the next billion years. Do you think such a situation is not really possible? Sure, but why? Because the $P(\mathrm{Heads})$ cannot suddenly change? But this sentence is meaningless for a P-frequentist.

I want to keep this answer short so I stop here; see above for the references. I think it is really difficult to be a die-hard P-frequentist.

(Update: In the comments below, @mpiktas insists that it is because the frequentist definition is mathematically meaningless. My opinion expressed above is rather that the frequentist definition is philosophically problematic.)

---

Frequentist approach to statistics

Consider a probabilistic model $P(X\mid\theta)$ that has some parameters $\theta$ and allows to compute the probability of observing data $X$. You did an experiment and observed some data $X$. What can you say about $\theta$?

S-frequentism is the position that $\theta$ is not a random variable; its true values in Real World are what they are. We can try to estimate them as some $\hat \theta$, but we cannot meaningfully talk about probability of $\theta$ being in some interval (e.g. being positive). The only thing we can do, is to come up with a procedure of constructing some interval around our estimate such that this procedure succeeds in encompassing true $\theta$ with a particular long-run success frequency (particular probability).

Most of the statistics used in natural sciences today is based on this approach, so there certainly are lots of S-frequentists around today.

(Update: if you look for an example of a philosopher of statistics, as opposed to practitioners of statistics, defending S-frequentist point of view, then read Deborah Mayo's writings; +1 to @NRH's answer.)

---

UPDATE: On the relationship between P-frequentism and S-frequentism

@fcop and others ask about the relationship between P-frequentism and S-frequentism. Does one of these positions imply another one? There is no doubt that historically S-frequentism was developed based on P-frequentist stance; but do they logically imply one another?

Before approaching this question I should say the following. When I wrote above that there are almost no P-frequentists I did not mean that almost everybody is P-subjective-bayesian-a-la-de-finetti or P-propensitist-a-la-popper. In fact, I believe that most statisticans (or data-scientists, or machine-learners) are P-nothing-at-all, or P-shut-up-and-calculate (to borrow Mermin's famous phrase). Most people tend to ignore foundation problems. And it is fine. We do not have a good definition of free will, or intelligence, or time, or love. But this should not stop us from working on neuroscience, or on AI, or on physics, or from falling in love.

Personally, I am not a S-frequentist, but neither do I have any coherent view on foundations of probability.

In contrast, almost everybody who did some practical statistical analysis is either a S-frequentist or a S-Bayesian (or perhaps a mixture). Personally, I published papers containing $p$-values and I have never (so far) published papers containing priors and posteriors over model parameters so this makes me a S-frequentist, at least in practice.

It is therefore clearly possible to be a S-frequentist without being a P-frequentist, despite what @fcop says in his answer.

Okay. Fine. But still: Can a P-bayesian be a S-frequentist? And can a P-frequentist be a S-bayesian?

For a convinced P-bayesian it is probably atypical to be a S-frequentist, but in principle entirely possible. E.g. a P-bayesian can decide that they do not have any prior information over $\theta$ and hence adopt a S-frequentist analysis. Why not. Every S-frequentist claim can certainly be interpreted with P-bayesian interpretation of probability.

For a convinced P-frequentist to be S-bayesian is probably problematic. But then it is very problematic to be a convinced P-frequentist...

Answer 2

Kolmogorov's work on Foundations of Theory of Probability has the section called "Relation to Experimental Data" on p.3. This is what he wrote there:

He's showing how one could deduct his Axioms by observing experiments. This is quite a frequentist way of interpreting the probabilities.

He has another interesting quote for impossible events (empty sets):

So, I think that if you're comfortable with these arguments, then you must admit that you're a frequentist. This label is not exclusive. You can be bi-paradigmous (I made up the word), i.e. both a frequentist and Bayesian. For instance, I become Bayesian when applying stochastic methods to phenomena which are not inherently stochastic.

UPDATE As I wrote earlier on CV, Kolmogorov's theory itself is not frequentist per se. It's as compatible with Bayesian view as with frequentist view. He put this cute footnote to the section to make very clear that he's abstaining from philosophy:

Answer 3

I believe that it is relevant to mention Deborah Mayo, who writes the blog Error Statistics Philosophy.

I won't claim to have a deep understanding of her philosophical position, but the framework of error statistics, as described in a paper with Aris Spanos, does include what is regarded as classical frequentist statistical methods. To quote the paper:

Under the umbrella of error-statistical methods, one may include all standard methods using error probabilities based on the relative frequencies of errors in repeated sampling – often called sampling theory or frequentist statistics.

And further down in the same paper you can read that:

For the error statistician probability arises not to measure degrees of confirmation or belief (actual or rational) in hypotheses, but to quantify how frequently methods are capable of discriminating between alternative hypotheses and how reliably they facilitate the detection of error.

Answer 4

Referring to this thread and the comments on it I think that the frequentists are those that define ''probability'' of an event as the long run relative frequency of the occurence of that event. So if $n$ is the number of experiments and $n_A$ the number of occurences of event $A$ then the probability of the event $A$, denoted by $P(A)$, is defined as $$P(A):=\lim_{n\to +\infty} \frac{n_A}n$$.

It is not hard to see that this definition fulfills Kolmogorov's axioms (because taking limits is linear, see also Is there any *mathematical* basis for the Bayesian vs frequentist debate?).

In order to give such a definition they must ''believe'' that this limit exists. So the frequentists are those who believe in the existence of this limit.

EDIT on 31/8/2016: on the distintion between S- and P-frequentism

As @amoeba distinguishes in his answer between S-frequentists and P-frequentists, where P-frequentists are the type of frequentists that I define supra, and as he also argues that it is hard to be a P-frequentist I added an EDIT section to argue that the opposite is true;

I argue that all S-frequentists are P-frequentists.

In the S-frequentism section @amoeba says ''this procedure succeeds in encompassing true $\theta$ with a particular long-run success frequency (particular probability).''

In his answer he also states that P-frequentists are a rare species.

But this ''long-run success frequency'', used to define S-frequentism, is what he defines as P-frequentism as it is the interpretation of $P(\widehat{CI} \ni \theta)$.

Therefore, according to his defintions every S-frequentist is also a P-frequentist. Therefore I conclude that P-frequentists are not so rare as argued by amoeba.

There is even more; @amoeba also argues that the S-frequentists consider the unknown parameter $\theta$ as fixed or non-random, therefore one can not talk about ''probability of $\theta$ having a particluar value'', he says that

''The only thing we can do, is to come up with a procedure of constructing some interval around our estimate such that this procedure succeeds in encompassing true $\theta$ with a particular long-run success frequency (particular probability).''

May I ask what might be the origin of the name ''frequentist'' : (a) the ''non-random $\theta$''-idea or (b) the ''long-run frequency''-idea ?

May I also ask @mpiktas who writes in his comment to the answer of amoeba:

'' It is very hard to be a P-frequentist, because it is practically impossible to give mathematically sound definition of such probability ''

If you need a defintion of P-frequentism to define the S-frequentism, how can one then be more S-frequentist than P-frequentist ?

Answer 5

Let me offer an answer that connects this question with a matter of current and very practical importance -- Precision Medicine -- while at the same time answering it literally as it was asked: Who are frequentists?

Frequentists are people who say things such as [1] (emphasis mine):

What does a 10% risk of an event within the next decade mean to the individual for whom it was generated? Contrary to what is thought, this risk level is not that person’s personal risk because probability is not meaningful in an individual context.

Thus, frequentists interpret 'probability' in such a way that it has no meaning in a singular context like that of an individual patient. My PubMed Commons comment on [1] examines the contortions its frequentist authors must undergo to recover a semblance of a probability-like notion applicable to the care of an individual patient. Observing how and why they do this may prove very instructive as to who is a frequentist. Also, the largely unilluminating subsequent exchange in the JAMA Letters section [2,3] is instructive as to the importance of explicitly recognizing limitations in frequentist notions of probability and attacking them directly as such. (I regret many CV users may find that [1] lies behind a paywall.)

The excellent and highly readable book [4] by L. Jonathan Cohen would repay the efforts of anyone interested in the OP's question. Of note, Cohen's book oddly was cited by [1] in connection with the claim "probability is not meaningful in an individual context," although Cohen clearly rebukes this view as follows [4,p49]:

Nor is it open to a frequency theorist to claim that all important probabilities are indeed general, not singular. It often seems very important to be able to calculate the probability of success for your own child’s appendectomy...

---

1] Sniderman AD, D’Agostino Sr RB, and Pencina MJ. “The Role of Physicians in the Era of Predictive Analytics.” JAMA 314, no. 1 (July 7, 2015): 25–26. doi:10.1001/jama.2015.6177. PubMed

2] Van Calster B, Steyerberg EW, and Harrell FH. “RIsk Prediction for Individuals.” JAMA 314, no. 17 (November 3, 2015): 1875–1875. doi:10.1001/jama.2015.12215. Full Text

3] Sniderman AD, D’Agostino Sr RB, and Pencina MJ. “RIsk Prediction for Individuals—reply.” JAMA 314, no. 17 (November 3, 2015): 1875–76. doi:10.1001/jama.2015.12221. Full Text

4] Cohen, L. Jonathan. An Introduction to the Philosophy of Induction and Probability. Oxford : New York: Clarendon Press ; Oxford University Press, 1989. Link to scanned pages 46-53 & 81-83

Answer 6

Really interesting question!

I'd put myself in the frequentist camp when it comes to understanding and interpreting probability statements, although I am not quite so hard-line about the need for an actual sequence of iid experiments to ground this probability. I suspect most people who don't buy the thesis that "probability is a subjective measure of belief" would also think about probability this way.

Here's what I mean: take our usual "fair" coin, with assignment $P(H)=0.5$. When I hear this, I form an image of someone tossing this coin many times and the fraction of heads approaches $0.5$. Now, if pressed, I would also say that the fraction of heads in any random sample from a finite sequence of such coin tosses will also approach $0.5$ as the sample size grows (independence assumption).

As has been stated by others, the biggest assumption is that this limit exists and is correct (i.e., limit is $0.5$), but I think just as importantly is the assumption that the same limit exists for randomly chosen sub-samples as well. Otherwise, our interpretation only has meaning wrt the entire infinite sequence (e.g., we could have strong autocorrelation that gets averaged out).

I think the above is pretty uncontroversial for frequentists. A Bayesian would be more focused on the experiment at hand and less on the long run behavior: they would state that their degree of belief that the next toss will be heads is $P(H) = 0.5$...full stop.

For a simple case such as coin tossing, we can see that the frequentist and Bayesian approaches are functionally equivalent, albeit philosophically very different. As Dikran Marsupial has pointed out, the Bayesian may in fact be utilizing the fact that empirically we see coins come up heads about as often as we see them come up tails (long run/large sample frequency as a prior).

What about things that cannot possibly have long run frequencies? For example, what is the probability North Korea will start a war with Japan in the next 10 years? For frequentists, we are really left in the lurch, since we cannot really describe the sampling distributions required to test such a hypothesis. A Bayesian would be able to tackle this problem by placing probability distribution over the possibilities, most likely based on eliciting expert input.

However, a key question comes up: where do these degrees of belief (or assumed value for the long run frequency) come from? I'd argue from psychology and say that these beliefs (especially in areas far from experimental data) come from what is referred to as the availability heuristic and representativness heuristic. There are slew of others that likely come into play. I argue this because in the absence of data to calibrate our beliefs (towards the observed long run frequency!), we must rely on heuristics, however sophisticated we make them seem.

The above mental heuristic thinking applies equally to Frequentists and Bayesians. What is interesting to me is that regardless of our philosophy, at the root, we place more belief in something that we think is more likely to be true, and we believe it to be more likely to be true because we believe there are more ways for it to be true, or we imagine that the pathways leading to it being true would happen more often (frequently:-) than those that would make it not true.

Since it's an election year, let's take a political example: What belief would we place in the statement "Ted Cruz will propose a ban assault rifles in the next 4 years". Now, we do have some data on this from his own statements, and we'd likely place our prior belief in the truth of this statement very near zero. But why? Why does his prior statements make us think this way? Because we think that highly ideological people tend to "stick to their guns" more than their pragmatist counterparts. Where does this come from? Likely from the studies done by psychologists and our own experiences with highly principled people.

In other words we have some data and the belief that for most cases where someone like Cruz could change their mind, they will not (again, a long-run or large-sample assessment of sorts).

This is why I "caucus" with the frequentists. It's not my dislike of Bayesian philosophy (quite reasonable) or methods (they're great!), but that if I dig deep enough into why I hold beliefs that lack strong large-sample backing, I find that I am relying on some sort of mental model where outcomes can be tallied (if implicitly) or where I can invoke long-run probabilities in a particular sub-process (e.g., Republicans vote against gun control measures X% of the time) to weight my belief one way or another.

Of course, this not really true frequentism, and I doubt that there are many people who subscribe to the von Mieses-esque interpretation of probability to the letter. However, I think it shows the underlying compatibility between Bayesian and Frequentist probability: Both are appealing to our inner heuristics regarding availability or what I call the "Pachinko" principle about frequencies along a chain of causation.

So perhaps I should call myself an "availabilist", to indicate that I assign probabilities based on how often I can imagine an event occurring as the outcome of a chain of events (with some rigor/modelling of course). If I have a lot of data, great. If I don't, then I will try to decompose the hypothesis into a chain of events and use what data I have (anecdotal or "common sense", as need be) to assess how often I would imagine such an event to occur.

Sorry for the longish post, great question BTW!

Answer 7

As @amoeba noticed, we have frequentist definition of probability and frequentist statistics. All the sources that I have seen until now say that frequentist inference is based on the frequentist definition of probability, i.e. understanding it as limit in proportion given infinite number random draws (as already noticed by @fcop and @Aksakal quoting Kolmogorov)

$$ P(A) = \lim_{n\to\infty} \frac{n_A}{n} $$

So basically, there is a notion of some population that we can repeatably sample from. The same idea is used in frequentist inference. I went through some classic papers, e.g. by Jerzy Neyman, to track the theoretical foundations of frequentist statistics. In the 1937 Neyman wrote

(ia) The statistician is concerned with a population, $\pi$, which for some reason or other cannot be studied exhaustively. It is only possible to draw a sample from this population which may be studied in detail and used to form an opinion as to the values of certain constants describing the properties of the population $\pi$. For example, it may be desired to calculate approximately the mean of a certain character possessed by the individuals forming the population $\pi$, etc.
(ib) Alternatively, the statistician may be concerned with certain experiments which, if repeated under apparently identical conditions, yield varying results. Such experiments are called random experiments [...]
In both cases described, the problem with which the statistician is faced is the problem of estimation. This problem consists in determining what arithmetical operations should be performed on the observational data in order to obtain a result, to be called an estimate, which presumably does not differ very much from the true value of the numerical character, either of the population $\pi$, as in (ia), or of the random experiments, as in (ib). [...]
In (ia) we speak of a statistician drawing a sample from the population studied.

In another paper (Neyman, 1977), he notices that the evidence provided in the data need to be verified by observing the repeated nature of the studied phenomenon:

Ordinarily, the 'verification', or 'validation' of a guessed model consists in deducing some of its frequentist consequences in situations not previously studied empirically, and then in performing appropriate experiments to see whether their results are consistent with predictions. Very generally, the first attempt at verification is negative: the observed frequencies of the various outcomes of the experiment disagree with the model. However, on some lucky occasions there is a reasonable agreement and one feels the satisfaction of having 'understood' the phenomenon, at least in some general way. Later on, invariably, new empirical findings appear, indicating the inadequacy of the original model and demanding its abandonment or modification. And this is the history of science!

and in yet another paper Neyman and Pearson (1933) write about random samples drawn from fixed population

In common statistical practice, when the observed facts are described as "samples," and the hypotheses concern the "populations", for which the samples have been drawn, the characters of the samples, or as we shall term them criteria, which have been used for testing hypotheses, appear often to be fixed by happy intuition.

Frequentist statistics in this context formalize the scientific reasoning where evidence are gathered, then new samples are drawn to verify the initial findings and as we accumulate more evidence our state of knowledge crystallizes. Again, as described by Neyman (1977), the process takes the following steps

(i) Empirical establishment of apparently stable long-run relative frequencies (or 'frequencies' for short) of events judged interesting, as they develop in nature.
(ii) Guessing and then verifying the 'chance mechanism', the repeated operation of which produces the observed frequencies. This is a problem of 'frequentist probability theory'. Occasionally, this step is labeled 'model building'. Naturally, the guessed chance mechanism is hypothetical.
(iii) Using the hypothetical chance mechanism of the phenomenon studied to deduce rules of adjusting our actions (or 'decisions') to the observations so as to ensure the highest 'measure' of 'success'. [...] the deduction of the 'rules of adjusting our actions' is a problem of mathematics, specifically of mathematical statistics.

Frequentists plan their research having in mind the random nature of data and the idea of repeated draws from fixed population, they design their methods based on it, and use it to verify their results (Neyman and Pearson, 1933),

Without hoping to know whether each separate hypothesis is true or false, we may search for rules to govern our behavior with regard to them, in following which we insure that, in the long run of experience, we shall not be too often wrong.

This is connected to repeated sampling principle (Cox and Hinkley, 1974):

(ii) Strong repeated sampling principle
According to the strong repeated sampling principle, statistical procedures are to be assessed by their behaviour in hypothetical repetitions under the same conditions. This has two facets. Measures of uncertainty are to be interpreted as hypothetical frequencies in long run repetitions; criteria of optimality are to be formulated in terms of sensitive behaviour in hypothetical repetitions.
The argument for this is that it ensures a physical meaning for the quantities that we calculate and that it ensures a close relation between the analysis we make and the underlying model which is regarded as representing the "true" state of affairs.

(iii) Weak repeated sampling principle
The weak version of the repeated sampling principle requires that we should not follow procedures which for some possible parameter values would give, in hypothetical repetitions, misleading conclusions most of the time.

As contrast, when using maximum likelihood we are concerned with the sample that we have, and in Bayesian case we make inference based on the sample and our priors and as new data appears we can perform Bayesian updating. In both cases the idea of repeated sampling is not crucial. Frequentists rely only on the data they have (as noticed by @WBT), but keeping in mind that it is something random and it is to be thought as a part of process of repeated sampling from the population (recall, for example, how confidence intervals are defined).

In frequentist case the idea of repeated sampling enables us to quantify the uncertainty (in statistics) and enables us to interpret real-life events in terms of probability.

---

As a side note, notice that neither Neyman (Lehmann, 1988), nor Pearson (Mayo, 1992) were as pure frequentists as we could imagine they were. For example, Neyman (1977) proposes using Empirical Bayesian and Maximum Likelihood for point estimation. On another hand (Mayo, 1992),

in Pearson's (1955) response to Fisher (and elsewhere in his work) is that for scientific contexts Pearson rejects both the low long-run error probability rationale [...]

So it seems that it is hard to find pure frequentists even among the founding fathers.

---

Neyman, J, and Pearson, E.S. (1933). On the Problem of the Most Efficient Tests of Statistical Hypotheses. Philosophical Transactions of the Royal Society A: Mathematical, Physical and Engineering Sciences. 231 (694–706): 289–337.

Neyman, J. (1937). Outline of a Theory of Statistical Estimation Based on the Classical Theory of Probability. Phil. Trans. R. Soc. Lond. A. 236: 333–380.

Neyman, J. (1977). Frequentist probability and frequentist statistics. Synthese, 36(1), 97-131.

Mayo, D. G. (1992). Did Pearson reject the Neyman-Pearson philosophy of statistics? Synthese, 90(2), 233-262.

Cox, D. R. and Hinkley, D. V. (1974). Theoretical Statistics. Chapman and Hall.

Lehmann, E. (1988). Jerzy Neyman, 1894 - 1981. Technical Report No. 155. Department of Statistics, University of Califomia.

Answer 8

"Frequentists vs. Bayesians" from XKCD (under CC-BY-NC 2.5), click to discuss:

The general point of the frequentist philosophy illustrated here is a belief in drawing conclusions about the relative likelihood of events based solely ("purely") on the observed data, without "polluting" that estimation process with pre-conceived notions about how things should or should not be. In presenting a probability estimate, the frequentist does not take into account prior beliefs about the likelihood of an event when there are observations available to support computation of its empirical likelihood. The frequentist should take this background information into account when deciding on the threshold for action or conclusion.

As Dikran Marsupial wrote in a concise comment below, "The valuable point the cartoon (perhaps unintentionally) makes is that science is indeed more complex and we can't just apply the "null ritual" without thinking about prior knowledge."

As another example, when trying to determine/declare what topics are "trending" on Facebook, frequentists would likely welcome the more purely algorithmic counting approach Facebook is shifting towards, instead of the old model where employees would curate that list based in part on their own background perspectives about which topics they thought "should" be most important.

Answer 9

(A remark, only tangentially relevant for the question and the site.)

Probability is about objective status of individual things. Things cannot have intention and they receive their statuses from the universe. With a thing, an event (giving it its status) always shall have happened: the event is already there accomplished, even if it haven't actually happened yet - the past future of a thing, also called "fate" or contingency.

Again, with probability, the fact of event - having yet occured or not, doesn't matter - is already there [as opposed to the meaning which never is there]; and as such it's already got unnecessary and superfluous. The fact should be discarded, and that invalidation of it is what we call "the event is probable". Any fact about a thing bears in itself its primeval unconvincing side, or probability of the fact (even the actually occured fact - we recognize it by pinprick of disbelief). We are inevitably "tired of things" pre-psychically to an extent. It remains therefore only to quantify that partial negation of facticity, if need a number. One way to quantify is to count. Another is to weigh. A frequentist carries out or imagines of a series of trials lying before him which he turns face over to see if the event actually happens; he counts. A Bayesian consideres a series of psychological motives dragging behing him which he screens; he weighs them as things. Both men are busy with charge/excuse game of mind. Fundamentally, there is no much difference between them.

Possibility is about potentialities of me in world. Possibility is always mine (a rain's chance is my problem to opt for taking an umbrella or getting wet) and concerns not an object (the one I'm considering as being possible or having the possibility) but the whole world for me. Possibility is always 50/50 and it is always convincing, because it implies - either calls for before or entails after - my decision how to behave. Things themselves have no intentions and thus possibilities. We should not confuse our possibilities of these things for us with their own probabilities of "stochastic determinism". Probability can never be "subjective" in the human sense.

---

An observant reader may feel in the response a masked dig at the bright answer in this thread, where @amoeba says he thinks "there are almost no frequentists of the [probability definition] kind (P-frequentists)". It could be turned opposite: bayesian probability definers do not exist as different class. Because, as I've admitted, bayesians consider chanks of reality the same manner as frequentists do - as series of facts; only these facts are not experiments, sooner recollections of "truths" and "arguments". But such forms of knowledge is factual and can only be counted or weighed. Probability it erects is not synthesized as subjective, that is, anticipatory ("bayesian" to be) unless human expectation (possibility) enters the scene to meddle. And @amoeba anxiously lets it in when imagines as "the coin will wear off and the Sun will go supernova".

Answer 10

Oh, I've been a frequentist for many's the year,
And I've spent all my time playing the data by ear,
But now I'm returning with Bayes in great store,
And I never will play the frequentist no more.

For it's no nay never, no nay never, no more,
Will I play the frequentist, no never, no more!

I went into a lab where I used to consult.
The gave me some data, said 'p that for us',
I said 'No way, Jose' with a bit of a smile,
P values and evident just don't reconcile!

Chorus

I said it's your prior that we need to shed light,
And the researcher's eyes opened wide with delight,
He said, 'My prior views are as good as the rest,
And for sure a Bayes factor is what will work best!'

Chorus

I'll go back to my teachers, confess what I've done,
And ask them to pardon their prodigal son,
But when they've forgave me, as often before,
I never will play the frequentist no more!

Chorus

And it's no, nay never, no nay never no more,
Will I play the frequentist, no never, no more!

Source: A E Raftery, in The Bayesian Songbook, edited by B P Carlin, at http://www.biostat.umn.edu/. Sung to the traditional folk tune of 'The Wild Rover'. Quoted in Open University M347 Mathematical Statistics, Unit 9.