There some words in PRML:

In a frequentist setting, w is considered to be a fixed parameter, whose value is determined by some form of ‘estimator’, and error bars on this estimate are obtained by considering the distribution of possible data sets D. By contrast, from the Bayesian viewpoint there is only a single data set D (namely the one that is actually observed), and the uncertainty in the parameters is expressed through a probability distribution over w.

I wonder what`s the meaning of "the distribution of possible data sets D". Why is it a distribution of data sets for frequentist?

  • $\begingroup$ Maybe it helps if you first consider a data set with only one column, 'male' say with $1=$ male and $0=$ female. $\endgroup$
    – Michael M
    Oct 5, 2015 at 14:58
  • $\begingroup$ I think this quotation may be a little misleading. It's a distribution of datasets for both schools of statistics. For the Bayesian, the probability distribution over $w$ is determined by a prior distribution over $w$, by the data, and by a model that associates every value of $w$ with a distribution of possible datasets. $\endgroup$
    – whuber
    Oct 5, 2015 at 19:49

1 Answer 1


Hello and welcome to stack exchange. As whuber pointed out, the perception that observed data is random is not specific to frequentist statistics. If we are modeling some process using random variables and then sample from that process, the sample we get will surely be random. Similarly, any conclusions that you make from the data are random, whether these are model parameters that were estimated or hypothesis test conclusions.

In frequentist statistics, the approach is to design experiments in such a way that making unacceptably wrong conclusions is acceptably improbable. Bayesians use probability distributions to model the uncertainty in their conclusions, and to incorporate prior beliefs rigorously into such a statistical approach.


Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge that you have read and understand our privacy policy and code of conduct.

Not the answer you're looking for? Browse other questions tagged or ask your own question.