The probability of a fair coin landing heads is $p(H)=1/2$ and since there is only one other outcome we can deduce that the probability for tails is also $p(T)=1-p(H)=1/2$. Yet if we examine a distribution of coin tosses, knowing just one parameter (for example heads) is not enough to describe the distribution. We also need to know either the number of tails or the total number of coin tosses, so how come there is only one degree of freedom in a coin toss distribution?
$\begingroup$
$\endgroup$
5
-
2$\begingroup$ Welcome to Cross Validated! It will be important to specify exactly what distribution you want to assess, as there are subtle differences between the so-called “Bernoulli” and “binomial” distributions. $\endgroup$– DaveCommented Aug 15, 2022 at 22:03
-
2$\begingroup$ The sample size (number of tosses here) is usually assumed to be known. $\endgroup$– HenryCommented Aug 15, 2022 at 22:09
-
$\begingroup$ @Henry does that not hide the point that you need two pieces of data to know the whole distribution? $\endgroup$– stinkyCommented Aug 15, 2022 at 22:22
-
4$\begingroup$ Please see stats.stackexchange.com/questions/16921 for what "degrees of freedom" can mean in statistics. (This is not one of those situations.) $\endgroup$– whuber ♦Commented Aug 15, 2022 at 22:43
-
$\begingroup$ Re "two pieces:" that's not true. One usually never knows the true distribution, no matter how much data are available. The principal exception would be when conducting a complete census (with no measurement error) of a finite population. And if by "know" you mean estimate, there are textbook examples where one observation suffices for an estimate. (In a Bayesian setting, one observation can give an excellent estimate.) $\endgroup$– whuber ♦Commented Aug 16, 2022 at 13:04
Add a comment
|
1 Answer
$\begingroup$
$\endgroup$
Degrees of freedom apply to a parametrisation of a model, not to the observed outcome. We model a single coin toss by a Bernoulli($p$) distribution, which only has a single parameter, namely the $p$. "Heads" is an outcome, not a parameter, so it neither has degrees of freedom nor is to be counted in order to know the degrees of freedom. If we want to estimate the parameter $p$ from data, say $n$ coin tosses, we usually assume the tosses to be i.i.d. (independently identically distributed) according to a Bernoulli($p$)-distribution, so there still is $p$ as the only parameter, as long as $n$ is known and treated as fixed (which usually is the case). The standard estimator is the relative frequency of heads (assuming "1" corresponds to "heads"), so we need to know this in order to estimate $p$, but it's an outcome, not a parameter to be estimated, so again it doesn't count on top of the $p$ for determining the number of parameters. You need to understand that there is an essential difference between the parameter $p$ to be estimated and the observed relative frequency used to estimate $p$.
answered Aug 15, 2022 at 22:15