9
$\begingroup$

I wonder whether conditional probabilities are unique to Bayesianism, or whether they are more of a general concept that is shared among several schools of thought among statistcs/probability people.

I kind of assume it is, because I assume that no one can $p(A,B) = p(A|B)p(B)$ is kind of logical, so I think frequentists would at least theoretically agree, while cautioning against Bayesian inference more out of practical reasons, and not because of conditional probabilities.

$\endgroup$
  • $\begingroup$ "Bayesian" and "frequentist" describe different approaches to solving problems, not different underlying theories. It took me a while to get this. Here is an example. $\endgroup$ – Mehrdad Jan 10 '18 at 23:35
  • 6
    $\begingroup$ I'd add that arguably all probabilities of any kind are conditional; it's just a case of whether the conditions are explicit, notationally or conceptually. $\endgroup$ – Nick Cox Jan 11 '18 at 10:54
  • $\begingroup$ Is this not simply a matter of the elements of an event sample space being either mutually exclusive and disjoint (independent) or otherwise joint (dependent)? Does not conditional probability derive from the latter? Therefore Bayesianism is just the special case of the application of a priori knowledge to derive the solution of a problem. $\endgroup$ – AsymLabs Jan 11 '18 at 12:47
  • $\begingroup$ The term "probability" is more restrictive in frequentist use than in Bayersian, so there are cases where p(A|B) and p(B) are valid frequentist probabilities, but p(A,B) is not. $\endgroup$ – Acccumulation Jan 11 '18 at 17:55
6
$\begingroup$

To pile up on the other and perfectly adequate answers, examples of conditional probability models abound in linear and generalised linear models since the definition of such models is conditional on the regressors or covariates: $$Y|X \sim f(y;g(X^\text{T}\beta),\sigma)$$

And the notion of conditional probability distributions is defined in measure theory with no reference to statistics and even less to "Bayesianism". For instance, Rényi built a probability theory out of conditional versions. Note also that in formal measure theory, conditioning is with respect to a $\sigma$-field $\mathfrak{S}$ rather than to an event. The conditional expectation $\mathbb{E}[X|\mathfrak{S}]$ is then a $\mathfrak{S}$-measurable function such that $$\mathbb{E}^{\mathfrak{S}}\{[X-\mathbb{E}[X|\mathfrak{S}]Z\}=0$$ for all $\mathfrak{S}$ measurable functions $Z$. (As illustrated by the concept of martingales.)

$\endgroup$
20
$\begingroup$

As with all probability theory, conditional probability has nothing to do with Bayesian vs frequentist statistics. Even Bayes' theorem is not “Bayesian”, but is a general theorem about probability, e.g. it can be used to correct probabilities for the base rate, without any priors, or subjective Bayesian interpretation for probability.

If you ask "what is the probability of getting the job of database engineer given that you are a female?", or "what is the probability that you have HIV given that the Western blot test was positive?", then you ask about conditional probabilities. Logistic regression models conditional probability, etc.

See also Is there any *mathematical* basis for the Bayesian vs frequentist debate? and Bayesian vs frequentist Interpretations of Probability

$\endgroup$
  • 2
    $\begingroup$ Could we use a less hot-button example? "The probability of running into an engineer that is less than 5'6" " for example. $\endgroup$ – JFA Jan 10 '18 at 21:40
  • 3
    $\begingroup$ @JFA I don’t see any problem with the example, at least it gives you a thought if conditioning makes sense in here. $\endgroup$ – Tim Jan 11 '18 at 6:16
10
$\begingroup$

Frequentist methods also use conditional probabilities. A p-value is a conditional probability. The only issue is that it is not a very useful or intuitive conditional probability. If we calculate a correlation coefficient and our machine spits out “p = .03,” what it is really saying is:

$p(D^*|H_0) = .03$

Where $D^*$ refers to the observed data or more extreme data (i.e., data that produces the observed result or a result stronger in the same direction) and $H_0$ is the null hypothesis (and all the assumptions that go along with it).

Conditioned on the null hypothesis, the probability we observe our data or more extreme data is .03. That’s a conditional probability completely absent of Bayes’ theorem. It’s just, in my opinion, usually not as useful (unless you are really trying to get at this probability for some reason or another).

$\endgroup$
  • 7
    $\begingroup$ I think "not intuitive" is a fair criticism, but "not useful" is a bit far. Criticisms of p-values are all well and good, but they can be put to good use by careful scientists. $\endgroup$ – Matthew Drury Jan 10 '18 at 19:50
  • 2
    $\begingroup$ @MatthewDrury that’s fair; I was too strong with my language. I have a publication record filled with inferences made off of p-values, so I suppose I have to agree. However, one could argue that the p-value inference is only useful insofar as it approximates Bayesian posterior coverage of zero, not in the inference per se. $\endgroup$ – Mark White Jan 10 '18 at 20:14
  • 4
    $\begingroup$ Yah, I agree that there is a reasonable argument to be made there. I just want us to be careful about our dismissiveness in our answers, its important to qualify. $\endgroup$ – Matthew Drury Jan 10 '18 at 21:07
  • $\begingroup$ @MatthewDrury +1 agreed and good point $\endgroup$ – Mark White Jan 10 '18 at 21:44
3
$\begingroup$

I don't think it's fair to say that conditional probabilities are unique to Bayesianism.

(Measure theory experts, please feel free to correct me.)

One way you could view a conditional probability - particularly when you have equally likely outcomes - is basing your probability calculation on a subset $\Omega^{\prime} \subset \Omega$, where $\Omega$ is the sample space.

For example, consider some fictitious data gathered (N.B.: we have no "prior" information) in a survey:

$$ \begin{array}{|l|c|c|} \hline & \text{Male} & \text{Female} \\ \hline \text{Owns a TV} & 75 & 72 \\ \text{Does not own a TV} & 25 & 28 \\ \hline \end{array}$$Let's assume that the probability of choosing any person surveyed above is equally likely. Consider the sample space $\Omega$ of all people surveyed and let $\mathbb{P} : \mathcal{A} \to [0, 1]$, where $\mathcal{A}$ is a non-empty $\sigma$-algebra of subsets of $\Omega$.

By definition of an equally likely event, for any event $A \in \mathcal{A}$, $$\mathbb{P}(A) = \dfrac{|A|}{|\Omega|}$$ where $|\cdot|$ denotes set cardinality.

If we were interested in, say, the probability of owning a TV given that you are a female, letting $A$ be the event of being a female and $B$ being the event of owning a TV, we would calculate the probability as $$\dfrac{|A \cap B|}{|A|}$$ and we're treating $A$ as our new sample space $\Omega^{\prime} = A$. But notice that we can write $$\dfrac{|A \cap B|}{|A|} = \dfrac{|A \cap B|/|\Omega|}{|A|/|\Omega|} = \dfrac{\mathbb{P}(A \cap B)}{\mathbb{P}(A)} $$ This is precisely the definition of conditional probability, and does not use Bayes' theorem. All we're doing is restricting our sample space.

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.