Bayesian vs frequentist Interpretations of Probability

Question

Can someone give a good rundown of the differences between the Bayesian and the frequentist approach to probability?

From what I understand:

The frequentists view is that the data is a repeatable random sample (random variable) with a specific frequency/probability (which is defined as the relative frequency of an event as the number of trials approaches infinity). The underlying parameters and probabilities remain constant during this repeatable process and that the variation is due to variability in $X_n$ and not the probability distribution (which is fixed for a certain event/process).

The bayesian view is that the data is fixed while the frequency/probability for a certain event can change meaning that the parameters of the distribution changes. In effect, the data that you get changes the prior distribution of a parameter which gets updated for each set of data.

To me it seems that the frequentist approach is more practical/logical since it seems reasonable that events have a specific probability and that the variation is in our sampling.

Furthermore, most data analysis from studies is usually done using the frequentist approach (i.e. confidence intervals, hypothesis testing with p-values etc) since it is easily understandable.

I was just wondering whether anyone could give me a quick summary of their interpretation of bayesian vs frequentist approach including bayesian statistical equivalents of the frequentist p-value and confidence interval. In addition, specific examples of where 1 method would be preferable to the other is appreciated.

Answer 1

In the frequentist approach, it is asserted that the only sense in which probabilities have meaning is as the limiting value of the number of successes in a sequence of trials, i.e. as

$$p = \lim_{n\to\infty} \frac{k}{n}$$

where $k$ is the number of successes and $n$ is the number of trials. In particular, it doesn't make any sense to associate a probability distribution with a parameter.

For example, consider samples $X_1, \dots, X_n$ from the Bernoulli distribution with parameter $p$ (i.e. they have value 1 with probability $p$ and 0 with probability $1-p$). We can define the sample success rate to be

$$\hat{p} = \frac{X_1+\cdots +X_n}{n}$$

and talk about the distribution of $\hat{p}$ conditional on the value of $p$, but it doesn't make sense to invert the question and start talking about the probability distribution of $p$ conditional on the observed value of $\hat{p}$. In particular, this means that when we compute a confidence interval, we interpret the ends of the confidence interval as random variables, and we talk about "the probability that the interval includes the true parameter", rather than "the probability that the parameter is inside the confidence interval".

In the Bayesian approach, we interpret probability distributions as quantifying our uncertainty about the world. In particular, this means that we can now meaningfully talk about probability distributions of parameters, since even though the parameter is fixed, our knowledge of its true value may be limited. In the example above, we can invert the probability distribution $f(\hat{p}\mid p)$ using Bayes' law, to give

$$\overbrace{f(p\mid \hat{p})}^\text{posterior} = \underbrace{\frac{f(\hat{p}\mid p)}{f(\hat{p})}}_\text{likelihood ratio} \overbrace{f(p)}^\text{prior}$$

The snag is that we have to introduce the prior distribution into our analysis - this reflects our belief about the value of $p$ before seeing the actual values of the $X_i$. The role of the prior is often criticised in the frequentist approach, as it is argued that it introduces subjectivity into the otherwise austere and object world of probability.

In the Bayesian approach one no longer talks of confidence intervals, but instead of credible intervals, which have a more natural interpretation - given a 95% credible interval, we can assign a 95% probability that the parameter is inside the interval.

Answer 2

You're right about your interpretation of Frequentist probability: randomness in this setup is merely due to incomplete sampling. From the Bayesian viewpoint probabilities are "subjective", in that they reflect an agent's uncertainty about the world. It's not quite right to say that the parameters of the distributions "change". Since we don't have complete information about the parameters, our uncertainty about them changes as we gather more information.

Both interpretations are useful in applications, and which is more useful depends on the situation. You might check out Andrew Gelman's blog for ideas about Bayesian applications. In many situations what Bayesians call "priors" Frequentists call "regularization", and so (from my perspective) the excitement can leave the room rather quickly. In fact, according to the Bernstein-von Mises theorem, Bayesian and Frequentist inference are actually asymptotically equivalent under rather weak assumptions (though notably the theorem fails for infinite-dimensional distributions). You can find a slew of references about this here.

Since you asked for interpretations: I think the Frequentist viewpoint makes great sense when modeling scientific experiments as it was designed to do. For some applications in machine learning or for modeling inductive reasoning (or learning), Bayesian probability makes more sense to me. There are many situations in which modeling an event with a fixed, "true" probability seems implausible.

For a toy example going back to Laplace, consider the probability that the sun rises tomorrow. From the Frequentist perspective, we have to posit something like infinitely-many universes to define the probability. As Bayesians, there is only one universe (or at least, there needn't be many). Our uncertainty about the sun rising is squelched by our very, very strong prior belief that it will rise again tomorrow.

Answer 3

The Bayesian interpretation of probability is a degree-of-belief interpretation.

A Bayesian may say that the probability that there was life on Mars a billion years ago is $1/2$.

A frequentist will refuse to assign a probability to that proposition. It is not something that could be said to be true in half of all cases, so one cannot assign probability $1/2$.

Answer 4

Chris gives a nice simplistic explanation that properly differentiates the two approaches to probability. But frequentist theory of probability is more than just looking at the long range proportion of successes. We also consider data sampled at random from a distribution and estimate parameters of the distribution such as the mean and variance by taking certain types of averages of the data (e.g. for the mean it is the arithmetic average of the observations. Frequentist theory associates a probability with the estimate that is called the sampling distribution.

In frequency theory we are able to show for parameters like the mean that are taken by averaging from the samples that the estimate will converge to the true parameter. The sampling distribution is used to describe how close the estimate is to the parameter for any fixed sample size n. Close is defined by a measure of accuracy (e.g. mean square error).

At Chris points out for any parameter such as the mean the Bayesian attaches a prior probability distribution on it. Then given the data Bayes' rule is used to compute a posterior distribution for the parameter. For the Bayesian all inference about the parameter is based on this posterior distribution.

Frequentists construct confidence intervals which are intervals of plausible values for the parameter. Their construction is based on the frequentist probability that if the process used to generate the interval were repeated many times for independent samples the proportion of intervals that would actually include the true value of the parameter would be at least some prespecified confidence level (e.g. 95%).

Bayesians use the a posteriori distribution for the parameter to construct credible regions. These are simply regions in the parameter space over which the posterior distibution is integrated to get a prespecified probability (e.g. 0.95). Credible regions are interpreted by Bayesians as regions that have a high (e.g. the prespecified 0.95) probability of including the true value of the parameter.

Answer 5

The following is taken from my manuscript on p-value functions - Johnson, Geoffrey S. "Decision Making in Drug Development via Inference on Power" Researchgate.net (2021).

In any quantitative field it is not enough to simply apply a set of mathematical operations. One must also provide an interpretation. The field of statistics concerns itself with a special branch of mathematics regarding probability. When interpreting probability there are primarily two competing paradigms: Bayesian and frequentist. These paradigms differ on what it means for something to be considered random and what probability itself measures. Both frequentists and Bayesians would agree that once a test statistic is observed it is fixed, there is nothing random about it. Additionally, frequentists and most Bayesians would agree that the parameter under investigation, say $\theta$, is an unknown fixed quantity and it is simply treated as random in the Bayesian paradigm as a matter of practice. The question then becomes, "How do we interpret probability statements about a fixed quantity?" Without delving into the mathematical details of how a posterior or a p-value is calculated, I explore various interpretations below and what makes them untenable.

One interpretation of a Bayesian prior is that "random'' is synonymous with "unknown'' and probability measures the experimenter's belief so that the posterior measures belief about the unknown fixed true $\theta$ given the observed data. This interpretation is untenable because belief is unfalsifiable $-$ it is not a verifiable statement about the actual parameter, the hypothesis, nor the experiment. Another interpretation is that "random'' is short for "random sampling'' and probability measures the emergent pattern of many samples so that a Bayesian prior is merely a modeling assumption regarding $\theta$, i.e. the unknown fixed true $\theta$ was randomly selected from a known collection or prevalence of $\theta$'s (prior distribution) and the observed data is used to subset this collection, forming the posterior distribution. The unknown fixed true $\theta$ is now imagined to have instead been randomly selected from the posterior. This interpretation is untenable because of the contradiction caused by claiming two sampling frames. The second sampling frame is correct only if the first sampling frame is correct, yet there can only be a single sampling frame from which we obtained the unknown fixed true $\theta$ under investigation. A third interpretation of a Bayesian prior is that "random'' is synonymous with "unrealized'' or "undetermined'' and probability measures a simultaneity of existance so that $\theta$ is not fixed and all values of $\theta$ are true simultaneously; the truth exists in a superposition depending on the data observed according to the posterior distribution (think Schrödinger's cat). This interpretation is untenable because it reverses cause and effect $-$ the population-level parameter depends on the data observed, but the observed data depended on the parameter. Ascribing any of these interpretations to the posterior allows one to make philosophical probability statements about hypotheses given the data. While the p-value is typically not interpreted in the same manner, it does show us the plausibility of a hypothesis given the data $-$ the ex-post sampling probability of the observed result or something more extreme if the hypothesis for the unknown fixed $\theta$ is true.

One might notice the similarity between a p-value and a posterior probability (or a confidence interval and a credible interval) and wonder under what circumstances is each one preferable. At its essence this is a matter of scientific objectivity. To the Bayesian, probability is axiomatic and measures the experimenter. To the frequentist, probability measures the experiment and must be verifiable. The Bayesian interpretation of probability as a measure of belief is unfalsifiable. Only if there exists a real-life mechanism by which we can sample values of $\theta$ can a probability distribution for $\theta$ be verified. In such settings probability statements about $\theta$ would have a purely frequentist interpretation. This may be a reason why frequentist inference is ubiquitous in the scientific literature. If the prior distribution is chosen in such a way that the posterior is dominated by the likelihood or is proportional to the likelihood, Bayesian belief is more objectively viewed as confidence based on frequency probability of the experiment.

In short, for those who subscribe to the frequentist interpretation of probability the p-value function summarizes all the probability statements about the experiment one can make as a function of the hypothesis for $\theta$. It is a matter of correct interpretation given the definition of probability and what constitutes a random variable. The posterior remains an incredibly useful tool and can be interpreted as an approximate p-value function.

Answer 6

From a "real world" point of view, I find one major difference between a frequentist and a classical or Bayesian "solution" that applies to at least three major scenarios. The difference in selecting a methodology depends on whether you need a solution that is impacted by the population probability, or one that is impacted by the individual probability. Examples below:

1. If there is a known 5% probability that males over 40 will die in a given year and require life insurance payments, an insurance company can use the 5% POPULATION percentage to estimate its costs, but to say that each individual male over 40 only has a 5% chance of dying ... is meaningless... Because 5% have a 100% probability of dying - which is a frequentist approach. At the individual level the event either occurs (100% probability) or it does not (0% probability) However, based on this limited information, it is not possible to predict the individuals who have a 100% probability of dying, and the 5% "averaged" population probability is useless at the individual level.

2. The above argument applies equally as well to fires in buildings which is why sprinklers are required in all buildings in a population.

3. Both of the above arguments apply equally as well to information systems breeches, damage, or "hacks". The population percentages are useless so all systems must be safeguarded.

Answer 7

“I was just wondering whether anyone could give me a quick summary of their interpretation of Bayesian vs. Frequentist approach including Bayesian statistical equivalents of the Frequentist p-value and confidence interval. In addition, specific examples of where one method would be preferable to the other are appreciated.”

At a certain level, you basically have it. However, I spent some time thinking about your question and thought I would add an answer to it.

The first thing I will do is change some language. When I speak about the Frequentist perspective, I will use the words data and parameters. When I speak about the Bayesian perspective, I will use the words observables and unobservables. Do notice that they are not the same. For example, if you are missing a data point, it is an unobservable but not a parameter in the Frequentist sense. Likewise, if it is known that the variance of a process is equal to some value $k$, then it is an observable. One of the difficulties of talking across paradigms is that people try and use the same words for almost identical concepts but are not quite the same.

In the general sense, you are correct. In the Frequentist view of the world, there are fixed but unknown parameters. Those fixed parameters determine what data could be seen. The data that could be seen is the sample space. I would note that when I say “fixed,” I do not necessarily mean that $\sigma^2=k$, it could be that $\sigma^2=k*t$ or any other function or possibly relation. I do not require homoskedasticity or that a distribution is stationary. I am saying that at any instantaneous moment, there is a parameter of fixed value. Those parameters are the determiners of that which is instantaneously possible.

Note that the long-run frequencies do not follow from the observations or their nature but the fixed nature of the natural system that the frequencies represent. An essential and exceedingly helpful side effect is that the sampling distributions of many estimators can be known. In the real world, we are often dealing with estimators rather than raw frequencies. Indeed, the raw frequencies are often of little use.

It is a bit more challenging to discuss the Bayesian view of the world because it has multiple axiomatizations. In contrast, there is only one axiomatization of the Frequentist side, Kolmogorov’s. Usually, the differences do not matter, especially in applied work. Nonetheless, they can matter in theoretical work. Savage’s and de Finetti’s solutions to probability differ in some theoretical constructions, as do others. That can result in differing applied Bayesian models, particularly in the social sciences.

In addition to axiomatic differences, there can be differences in interpretation created by the silences in the math. Bayesian theory often does not require you to adopt a particular point of view, but we are humans, and we like to feel comfortable. I suspect people that work in quantum mechanics have the same difficulty.

As an example, there are two equally valid ways that you could view a prior distribution for an unobservable. The first would be that the unobserved quantity is fixed, but its location is unknown. The prior represents your uncertainty about its location. It is a representation of your beliefs about it. The second would be that Nature draws the actual value of the unobservable when you do an experiment from a distribution called the prior. Your prior represents your best estimate of what nature is doing. They are equivalent; to reject one is to deny the other. You can emotionally reject one, but not the math. You can assert one of the two, but that is a statement of emotional comfort, not math. The math doesn’t make nature do anything.

Randomness on the Bayesian side of the coin is uncertainty. There are unobservable things that you want to know more about, or to take actions about or because of, likely based on observable things. You are uncertain about the unobservables. You are certain of the observables. Please note that just because you are certain, treating the observables as fixed in the same sense as Frequentist parameters, does not imply that your observations are valid. If you observe a magician providing you with data, the observations are fixed. It does not mean that they are accurately informative of an underlying phenomenon.

Regardless, Bayesian probability measures and statistics are subjective. They depend entirely upon the prior knowledge of the system. A seasoned engineer with twenty years of experience and a graduate degree will have a different prior regarding soil samples for constructing a bridge than a fresh-out engineer who graduated engineering school last month. That difference in skill and knowledge can have very real-world consequences. A new engineer happily accepting the results of a t-test may find a grumpy senior engineer requiring more sampling and a rejection of the results by the inclusion of his or her prior.

Bayesian methods are about updating beliefs. The probability distributions are the distributions of belief. That may imply that there would be no scientific use, but that is not true. If one were to adopt the prior of an ardent opponent and show that even a highly bigoted opponent should assume the opposite belief, then that is very convincing. A passionate proponent of using Ivermectin to treat COVID, as long as they do not have a degenerate prior conviction that there is only one answer, will give up on Ivermectin as data comes through. It may take much longer than for a person with no personal opinion one way or the other, but it will happen. To be honest, because doctors prescribed so much Ivermectin in the last few months to keep their patients from going to other doctors, there is now an extensive data set. We have data from controlled experiments and natural experiments. The upshot of this is that people should get vaccinated and seek other treatments such as the monoclonal or polyclonal antibody infusions early in the infection.

As long as your prior beliefs are not degenerate, they can change upon seeing data, then the data will drive you to reality. Subject to you not living in the Truman show or gathering your entire worldview from magicians and con artists, the data wins eventually.

As to Bayesian equivalents to the p-value or the confidence interval, there are none. A p-value provides the probability of observing a result as extreme or more extreme if the null hypothesis is true. There can be no real Bayesian equivalent because there is no equivalent to the null hypothesis on the Bayesian side. No hypothesis is special, and there is no restriction to a null and an alternative. You can have any finite number of hypotheses that you find meaningful.

The closes thing is the Bayesian posterior probability. It is a statement of how much weight you give to the truth of a hypothesis. It has nothing to do with chance. The hypothesis is not assumed to be true. The question the posterior resolves is what probability do you give or how much credence or credibility do you provide a hypothesis.

There is no Bayesian equivalent to a confidence interval. A confidence interval is any function that guarantees that the interval will cover a parameter to some specified percentage of the time. There is an infinite number of such functions. Confidence intervals are not unique. Suppose you repeat an experiment an infinite number of times. In that case, the percentage of time that your interval will cover the parameter will never be less than your desired guaranteed percentage.

Of course, since infinite repetition isn’t feasible, it is just a model, as are Bayesian models.

Of great importance, if you perform an experiment and a parameter is estimated with 95% confidence to be in the interval $[a,b]$, that does not imply that there is a 95% chance that the parameter is in the interval. It is a statement of confidence in the interval building process. You believe that at least 95% of your intervals will cover the parameter as the number of experiments goes to infinity.

The closest Bayesian equivalent is the credible set. It is not an interval, and it does not have to be a connected set. It can certainly be true that the Frequentist confidence interval is $[5,15]$, when the Bayesian set of equivalent probability is $[6,7]\cup[8,9.5]$. The set can be disjoint if some area is improbable. As there is an infinite number of ways to subset a probability distribution, there is an endless number of possible credible sets that all add to at least some chosen percentage.

The credible interval is created by applying some rule to the posterior probability distribution. So a 95% credible set is the region where you would give at least a 95% probability of finding the parameter given your observations and prior knowledge.

Which method you should use depends entirely upon what usage you are going to put the techniques to.

Fisher’s method of maximum likelihood should be used whenever you want to acquire new knowledge. If you wonder if something is true, then you collect data and research it. Take that data, plug it into the method of maximum likelihood, and use that to generate a p-value or a likelihood ratio. If the p-value is small enough, then provisionally accept that your null hypothesis is false and do more research into the topic. If it is not small enough for you, then realize that you have wasted your time and go on to other, hopefully, more fruitful, things.

Pearson and Neyman’s frequency method should be used when it matters whether you accept or reject something. It allows you to create an acceptance and rejection region and gives you a way to control for statistical power. An excellent example of that would be quality control inspection. The method says that if you choose some value, $\alpha$, and stick to it, then you will be made a fool of no more often than $\alpha$ percentage of the time.

Laplace’s method of inverse probability, Bayesian analysis, should be used when you need to find something, gamble, take personal action, or update your beliefs. You should never ever place money at risk other than with Bayesian methods unless you want someone to have the ability to force you to take sure losses. Risk-taking with money is my area, so it colors my perspective. Likewise, if you need to find a downed plane, use a Bayesian method. If you need to find an unobservable quantity with observable data, Bayes is your tool. If you need to be able to make factual statements about a parameter using conventions that we can all agree with, Frequentist or Likelihoodhist methods are your toolkit.

That boils it down. The frequency side answers, “what are the minimal statements we can all agree with because we can have statistical confidence in the procedures that were used to create it?” The Bayesian side answers, “what should I believe, or how should I act, based only on what I saw and prior observations and the prior observations of other people that I have chosen to endorse?”

Answer 8

The choice of interpretation depends on the question. If you wish to know the odds in a game of chance, classical interpretation will solve your problem, but statistical data is useless since fair dice have no memory.

If you wish to predict a future event based on past experience, the frequentist interpretation is correct and sufficient.

If you don't know if a past event had occurred, and wish to assess the probability that it did, you must take your prior beliefs, i.e. what you already know about the chance of the event to occur and update your belief when you acquire new data.

Since the question is about a degree of belief, and each person may have a different idea about the priors, the interpretation is necessarily subjective, a.k.a. Bayesian.

Answer 9

The other answers do a good job explaining this topic, but I think there's room for a more motivated explanation.

What is the probability of rolling a 1 with a fair six-sided die? The traditional answer would be 1 in 6, because no one side is favored over another. This concept can be extended to the principle of indifference, which was stated by Keynes (1921):

If there is no known reason for predicating of our subject one rather than another of several alternatives, then relatively to such knowledge the assertions of each of these alternatives have an equal probability. Thus equal probabilities must be assigned to each of several arguments, if there is an absence of positive ground for assigning unequal ones.

And so we find the probability of pulling any card from a deck is 1 in 52, the probability of flipping heads with a fair coin is 1 in 2, the probability of pulling a 5 from a random number generator which gives single digit integers from 0 to 9 is 1 in 10. We know that, with sufficient information about the system, the result could always be predicted exactly, and so there is no "chance" involved, only lack of complete knowledge. But unless we have evidence that a particular outcome is more likely, we must consider all equally.

We can now distinguish two types of probability:

• Subjective probability, which is the degree of belief of an actor based on a priori knowledge
• Objective probability, the theoretical chance or propensity of an event

Presumably, our goal is to better define the concept of objective probability, or to limit ourselves to subjective probabilities through which we infer something like the objective probability. In both cases, we wish to develop a model for the events of interest. We will work towards understanding probability from the perspective of Bayesion probability (subjective probability based on observations) and frequentist probability (objective probability based on the idea that, in the limit, the relative frequencies obtained by random sampling give the probability of the respective events). But first, we must give evidence that the principle of indifference is flawed if misapplied.

The principle of indifference is subject to paradoxes, one of which is Bertrand's paradox (see e.g. here, slightly modified). It goes as follows:

Consider an equilateral triangle inscribed in a circle. Suppose a chord of the circle is chosen at random. What is the probability that the chord is longer than a side of the triangle?

A key idea here is that the domain of possibilities is infinite, there are an infinite number of possible chords we can choose, and many ways we can "randomly" select them. For example:

1. Random endpoints - Select two random points on the circumference of the circle and connect them; the probability that the chord is longer than the side of an equilateral triangle inscribed in the circle is 1 in 3
2. Random radials - Select a radius line of the circle, choose a point on the radius, and construct the chord through this point and perpendicular to the radius. The probability of the chord being longer than a side is 1 in 2.
3. Random midpoints: Select a random point as the midpoint of a chord. The probability that the chord is longer is 1 in 4.

There are other considerations (e.g. the role of diameter lines, the "maximum ignorance" principle), but by the classical theory of probability, the problem as stated above has no unique solution. While Bertrand's paradox is still considered by some as unresolved, whether you believe that or not the paradox still illustrates issues with naively adapting the principle of indifference.

Another problem in probability which the traditional interpretation of probability fails to answer is this: what is the probability that the sun will rise tomorrow? There is no natural symmetry by which to reason, so the traditional approach is difficult to apply. The question can be interpreted in different ways, but we seek to answer what the probability should be. As an example, I [or humans in general] have experienced the sun rising every day since I have been alive, therefore my degree of belief that the sun will rise tomorrow is very high. Alternatively, the physical mechanism behind the sun "rising" is well-known, and no observations have been made which would lead us to think that the Earth will stop rotating, so there is a very small probability that the sun will not rise, the only plausible reason for denying this being a lack of absolute information about all possible astronomical events in the next 24 hours.

Laplace first posed this problem, and describes the probability as follows. Assume the sun rises on $p\%$ of days, where $p$ can only be inferred from experience. We wish to calculate the probability that $p$ is in a given range, based on the number of days we have actually seen the sun rise. Initially (at the beginning of Earth time, say), we have no information about $p$, therefore we assume a uniform distribution from 0 to 100%. That is, the probability that the sun will rise can be anywhere in the range $[0,100\%]$ with equal likelihood. (Careful to note: that's the probability of a probability.)

Now, every day on record where the sun has risen is evidence in favor of the statement "the sun will rise tomorrow." Therefore, the distribution of $p$ is calculated from the conditional probability of the sun rising tomorrow given that the sun has risen $k$ times previously. The initial distribution (the uniform distribution) of the probability $p$ is called the a priori distribution, or prior distribution. From Bayes' rule, this conditional probability can be calculated from the prior distribution and the observations.

Taking this idea further, the Bayesian interpretation of probability states that any probability is a conditional probability given knowledge about the population. Bayesian probability frames problems in e.g. statistics in quite a different way, which the other answers discuss.

The Bayesian system seems to be a direct application of the theory of probability, which seeks to avoid inferring anything which is not already known, and only inferring based on exactly what has been observed. Initially, it was only considered within the domain where this kind of reasoning was desirable (like the sunrise problem), while problems with random sampling and statistical inference were considered in their own domain. But, in the mid-20th century, the assumptions of the statistical approach of random sampling began to be understood. Most of the early 20th century statistical techniques followed an approach to probability similar to the following: given an exactly defined random experiment, which can be repeated without subjectivity, one may estimate the probability of the event occurring in general, and as the number of observations increases, the objective probability of the event is approached. The fact that this was an alternative interpretation of the concept of probability was not immediately obvious, but it grew to be well accepted. Because of the role of relative frequencies of events occurring, this was described as the frequentist interpretation of probability.

Returning to the sunrise problem. From the Bayesian perspective, we began with a prior distribution (uniform probability) for the probability $p$ that the sun will rise tomorrow, and we used the repeated experiences of the sun rising, combined with Bayes' rule, to obtain a conditional probability. The probability is only what we can obtain by starting with ignorance and working towards understanding.

From the frequentist perspective, the experiment is not the well-defined random experiment necessary, so we cannot directly apply the concept. But if we want to force it, we also consider the past experiences, considering these as a sample of the sample space (all times the sun will or will not rise in the morning). With an imperfect understanding of the experiment and underlying mechanism, we would be forced to infer that because the sun has always risen in the past, that it will always continue to rise in the future, within some confidence interval. This problem clearly favors a Bayesian approach.

Compare this with any of the random experiments featured in statistical textbooks (random number generators, whether a die is loaded or fair, presence of genetic traits in plant or animal populations). Before the rise of modern computing, calculating the Bayesian probability of events was almost impossible. But in the last few decades, Bayesianism has seen a marked rise in popularity, especially in problems where it is most applicable (as to which problems those are, sometimes it's clear, but generally it's an open problem).