4
$\begingroup$

This is on page 133 of the book: https://www.deeplearningbook.org/contents/ml.html#pf10 enter image description here

In the above, it says that

the data set is directly observed and so is not random

If that data we observe is not random, wouldn't the probability of the data occurring equal to 1. And thus, the denominator of Bayes' theorem also be 1? That should be incorrect as we wouldn't be able to normalize the numerator (a likelihood) if we divide it by 1...

In Bayesian stats, are both the data AND parameters of the data generating process both considered random? The book seems to imply that the data is fixed. In Frequentist, seems like only the data is viewed to be random.

Improve this question
$\endgroup$
  • 5
    $\begingroup$ Bayesian statistics has a bunch of different interpretations, so I don't want to claim to speak for all Bayesians. But the view taken in this book is that Bayesian statistics conditions on the things you know. Since the data is known, you should condition on it, so it is no longer random. If I flip a coin and observe "Heads" then there is nothing uncertain about the result any longer; the result was uncertain before flipping, but not after. This doesn't contradict Bayes rule, because the denominator uses the unconditional distribution of the (observed/"fixed") data. $\endgroup$ – guy Mar 19 '20 at 23:02
4
$\begingroup$

This quote is a bit of an ambiguous statement if not a typo per se. Bayesian statistics fully exploits the fact that the data $x$ is the realisation of a random variable, $X=x$, with $$X\sim f(x|\theta)$$by actualising the prior distribution $\pi(\theta)$ into the posterior $$\pi(\theta|x) = \dfrac{\pi(\theta)f(x|\theta)}{\int_\Theta\pi(\theta)f(x|\theta)\,\text{d}\theta}$$ It would thus have been clearer to avoid such a statement and replace it with

the data set is directly observed and so is conditioned upon

Improve this answer
$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.