Why need frequentist considered "a distribution of possible data sets D" in likelihood function p(D|w)?

Question

There some words in PRML:

In a frequentist setting, w is considered to be a fixed parameter, whose value is determined by some form of ‘estimator’, and error bars on this estimate are obtained by considering the distribution of possible data sets D. By contrast, from the Bayesian viewpoint there is only a single data set D (namely the one that is actually observed), and the uncertainty in the parameters is expressed through a probability distribution over w.

I wonder what`s the meaning of "the distribution of possible data sets D". Why is it a distribution of data sets for frequentist?

Answer 1

Hello and welcome to stack exchange. As whuber pointed out, the perception that observed data is random is not specific to frequentist statistics. If we are modeling some process using random variables and then sample from that process, the sample we get will surely be random. Similarly, any conclusions that you make from the data are random, whether these are model parameters that were estimated or hypothesis test conclusions.

In frequentist statistics, the approach is to design experiments in such a way that making unacceptably wrong conclusions is acceptably improbable. Bayesians use probability distributions to model the uncertainty in their conclusions, and to incorporate prior beliefs rigorously into such a statistical approach.