How do frequentists guess a distribution?

With competing hypotheses such as testing if a coin is fair, frequentists and Bayesians have their own approaches.

What about for coming up with a distribution? In An Essay towards solving a Problem in the Doctrine of Chances, Bayes guesses a posterior distribution through choosing a particular distribution (prior) and then flipping a few times.

What about for frequentists? What is their approach to that particular problem if they do not choose a distribution?

Part of a series in trying to understand Bayesian inference 1 2 3.

• I don't understand the distinction you're making. Could you articulate it a little more precisely? In either paradigm, the test that "a coin is fair" is mathematically framed in terms of inference about which Bernoulli distribution describes its outcomes. That already amounts to "coming up with a distribution." There would appear to be no difference at all, then, between the situations you describe in the first two paragraphs.
– whuber
Aug 11 '15 at 15:08
• In the same way Bayes did. A modern "frequentist" does not eschew Bayes' theorem--and in fact makes extensive use of it. Indeed, where a prior distribution can be proposed and defended, a frequentist has no problem whatsoever in using it and updating it according to all the rules of Bayesian analysis. Where the two paradigms part company is (1) a frequentist will require stronger support for the prior--it is unacceptable that it be "subjective"--and (2) the frequentist can avail herself of many procedures considered by Bayesians to be invalid. A modern "frequentist" is really a pragmatist.
– whuber
Aug 11 '15 at 18:41
• Glen_b has provided a good answer. I would add that you seem to be equating "frequentists" with "people who only conduct hypothesis tests." There are many other statistical procedures available to learn about data. For example, a (knowledgeable) frequentist would select procedures to estimate $p$ based on their risk functions if a prior for $p$ were not available or were suspect. A (knowledgeable) Bayesian effectively does the same thing by evaluating the sensitivity of their result to the prior. Perhaps they differ more in how they conceive of and interpret $p$.
– whuber
Aug 12 '15 at 11:51
• The Wikipedia article obviously is limited: it neglects estimation and prediction in particular.
– whuber
Aug 13 '15 at 0:51
• I don't know what you have learned, so I can't comment on it. You might be interested to know, though, that many of the classical procedures taught in classes have their justification through Bayes' theorem, which is a fundamental tool used to demonstrate admissibility of a procedure (which means the procedure is not consistently outperformed by some other possible procedure). In fact, the argument that OLS estimators are inadmissible(!) is made in this fashion.
– whuber
Aug 13 '15 at 13:54

What that article claims Bayes came up with was a prior distribution for $p$.

A frequentist doesn't have a distribution on $p$ (though see the note below about incorporating previous data for example - a frequentist is not bound to ignore other information on the parameter).

So the frequentist doesn't "guess" a distribution for $p$. In fact, neither does the Bayesian - you don't guess your prior on a parameter, as a Bayesian you choose* it (along the lines of "given what I already understand about this situation that generates successes and failures, what can I say about where $p$ is likely to be?"; there using likely in its ordinary sense).

* I'm glossing over differences between some flavors of Bayesianism here.

Given the same assumptions about the situation (presumably Bernoulli trials of some kind), both of them would have the same model for the number of successes in n trials, given some $p$ (a binomial), arrived at via the same reasoning (the progression from a sequence of Bernoulli trials with constant $p$ to a Binomial number of successes in $n$ trials is straightforward application of probability rules that both agree on).

So they agree how to model (say) coin-flips for a given probability of a head ($p$), and they generally agree on the relevance of the likelihood for the relationship between the data, the model for the data and the information about the parameter(s). The trick is to apply that agreed-on model for the data to a situation where you want to come to infer information about $p$.

They differ on how $p$ is treated. The frequentist treats $p$ as fixed but unknown and tries to get information about it via things like point estimates, confidence intervals with certain coverage probabilities, and so on. The Bayesian treats their uncertainty about $p$ as represented by a probability distribution, which the data then narrows down, via (say) a credible interval (though I'm leaving out some stuff that's important to many Bayesians here). While in many situations a credible interval and a confidence interval look very similar (or with particular choices of prior, even identical), they're not trying to achieve the same thing.

[If you have information relating to your particular $p$ garnered from prior data (e.g. yesterday's experiment with the same coin), the Bayesian and the frequentist tend to agree how to incorporate that prior data with the current data; they're both applying the same probability rules there.]

The use of a uniform distribution for the prior on $p$ there was presumably intended to represent "total ignorance" of $p$, but always using flat priors to represent "ignorance" leads to an interesting situation -- your inference then depends on how you parameterize the situation. If person A works with the probability, $p$, while B works with the odds ratio $\omega=\frac{p}{1-p}$, and person C uses the log-odds ($\eta=\log \omega$), then when they come to convert their knowledge of the parameters to their friends' parameterizations, they will come to (at least slightly) different conclusions about the parameter.

(There are priors that don't depend on how you parameterize.)

• I just edited again, sorry. There's lots of details that I have to leave out in an exposition of this brevity that lead to this requiring some caveats. I'm still not quite happy with it, to be honest. Aug 12 '15 at 1:36
• So essentially 'The frequentist treats p as fixed but unknown and tries to get information about it via things like point estimates, confidence intervals with certain coverage probabilities, and so on. The Bayesian treats their uncertainty about p as represented by a probability distribution' ?
– BCLC
Aug 13 '15 at 0:40
• If I understand you, I believe the parts of your question you changed is pretty much already covered by my answer and whuber's initial comment. Bayesians don't "guess" a posterior, and nor did Bayes (this is the point of the first sentence of my answer). They choose a prior and update it by multiplying by the likelihood (in the coin example based on a Bernoulli model for flipping a coin as whuber mentioned, also discussed in my answer). A frequentist will use the same Bernoulli model and so the same likelihood function, and they don't have priors or posteriors in the same sense. ... ctd Sep 7 '15 at 13:22
• ctd ... I am not sure where we're failing to communicate but perhaps we're talking at cross purposes. Sep 7 '15 at 13:23
• I wouldn't put it that way around but in sense yes Sep 18 '15 at 15:58