Conditional probabilities - are they unique to Bayesianism?

Question

I wonder whether conditional probabilities are unique to Bayesianism, or whether they are more of a general concept that is shared among several schools of thought among statistcs/probability people.

I kind of assume it is, because I assume that no one can $p(A,B) = p(A|B)p(B)$ is kind of logical, so I think frequentists would at least theoretically agree, while cautioning against Bayesian inference more out of practical reasons, and not because of conditional probabilities.

Answer 1

To pile up on the other and perfectly adequate answers, examples of conditional probability models abound in linear and generalised linear models since the definition of such models is conditional on the regressors or covariates: $$Y|X \sim f(y;g(X^\text{T}\beta),\sigma)$$

And the notion of conditional probability distributions is defined in measure theory with no reference to statistics and even less to "Bayesianism". For instance, Rényi built a probability theory out of conditional versions. Note also that in formal measure theory, conditioning is with respect to a $\sigma$-field $\mathfrak{S}$ rather than to an event. The conditional expectation $\mathbb{E}[X|\mathfrak{S}]$ is then a $\mathfrak{S}$-measurable function such that $$\mathbb{E}^{\mathfrak{S}}\{[X-\mathbb{E}[X|\mathfrak{S}]Z\}=0$$ for all $\mathfrak{S}$ measurable functions $Z$. (As illustrated by the concept of martingales.)

Answer 2

As with all probability theory, conditional probability has nothing to do with Bayesian vs frequentist statistics. Even Bayes' theorem is not “Bayesian”, but is a general theorem about probability, e.g. it can be used to correct probabilities for the base rate, without any priors, or subjective Bayesian interpretation for probability.

If you ask "what is the probability of getting the job of database engineer given that you are a female?", or "what is the probability that you have HIV given that the Western blot test was positive?", then you ask about conditional probabilities. Logistic regression models conditional probability, etc.

See also Is there any *mathematical* basis for the Bayesian vs frequentist debate? and Bayesian vs frequentist Interpretations of Probability

Answer 3

Frequentist methods also use conditional probabilities. A p-value is a conditional probability. The only issue is that it is not a very useful or intuitive conditional probability. If we calculate a correlation coefficient and our machine spits out “p = .03,” what it is really saying is:

$p(D^*|H_0) = .03$

Where $D^*$ refers to the observed data or more extreme data (i.e., data that produces the observed result or a result stronger in the same direction) and $H_0$ is the null hypothesis (and all the assumptions that go along with it).

Conditioned on the null hypothesis, the probability we observe our data or more extreme data is .03. That’s a conditional probability completely absent of Bayes’ theorem. It’s just, in my opinion, usually not as useful (unless you are really trying to get at this probability for some reason or another).

Answer 4

I don't think it's fair to say that conditional probabilities are unique to Bayesianism.

(Measure theory experts, please feel free to correct me.)

One way you could view a conditional probability - particularly when you have equally likely outcomes - is basing your probability calculation on a subset $\Omega^{\prime} \subset \Omega$, where $\Omega$ is the sample space.

For example, consider some fictitious data gathered (N.B.: we have no "prior" information) in a survey:

$$ \begin{array}{|l|c|c|} \hline & \text{Male} & \text{Female} \\ \hline \text{Owns a TV} & 75 & 72 \\ \text{Does not own a TV} & 25 & 28 \\ \hline \end{array}$$Let's assume that the probability of choosing any person surveyed above is equally likely. Consider the sample space $\Omega$ of all people surveyed and let $\mathbb{P} : \mathcal{A} \to [0, 1]$, where $\mathcal{A}$ is a non-empty $\sigma$-algebra of subsets of $\Omega$.

By definition of an equally likely event, for any event $A \in \mathcal{A}$, $$\mathbb{P}(A) = \dfrac{|A|}{|\Omega|}$$ where $|\cdot|$ denotes set cardinality.

If we were interested in, say, the probability of owning a TV given that you are a female, letting $A$ be the event of being a female and $B$ being the event of owning a TV, we would calculate the probability as $$\dfrac{|A \cap B|}{|A|}$$ and we're treating $A$ as our new sample space $\Omega^{\prime} = A$. But notice that we can write $$\dfrac{|A \cap B|}{|A|} = \dfrac{|A \cap B|/|\Omega|}{|A|/|\Omega|} = \dfrac{\mathbb{P}(A \cap B)}{\mathbb{P}(A)} $$ This is precisely the definition of conditional probability, and does not use Bayes' theorem. All we're doing is restricting our sample space.

Answer 5

I'm a bit late to this particular party, but I figured I would add a more philosophical answer to the other excellent answers here, in case it might be helpful for future searchers.

If you are a hypothetical frequentist, then the definition of conditional probability follows from the limit law for division. Explicitly, let $f_N (A \land E)$ be the number of times $A\land E$ is true in $N$ trials and let $f_N (E)$ be the number of times $E$ is true in $N$ trials. We define $$ p(A\land E) := \lim_{N\to \infty} \frac{f_N (A\land E)}{N} $$ and $$ p (E) := \lim_{N\to \infty} \frac{f_N (E)}{ N} $$ Finally, let $p(A | E )$ be the fraction of the times when $E$ is true that $A$ is also true, in the infinite limit: $$ p (A | E) := \lim_{N\to \infty} \frac{f_N (A\land E)}{f_N (E)} $$ Supposing $p(E)$ is non-zero, we have $$ p (A | E) = \lim_{N\to \infty} \frac{f_N (A\land E)/ N}{f_N (E)/N} = \frac{\lim_{N\to \infty} f_N (A\land E) / N}{\lim_{N\to \infty} f_N (E)/ N } = \frac{p(A\land E)}{p(E)}. $$