7
$\begingroup$

Following simple example from Wikipedia's definition of sufficient statistic with Bernoulli distribution with parameter $\theta$, where sufficient statistic is a sum of successes $$T(X_n)=\sum_{i=1}^n x_i, x_i \in \{0,1\}$$ As MLE is $\frac{1}{n}\sum_{i=1}^n x_i$ we can't just use the sufficient $T(x)$, but we also need the sample size, which is a property of our sample, hence $T(x)$ doesn't contain all information about the sample. Is there an unspoken rule, that the sample size is not considered for the sufficient statistics?

$\endgroup$
5
  • $\begingroup$ This is a good question (+1), probably because the sample size is considered a known constant, and not a random variable. $\endgroup$ Jun 28, 2022 at 18:05
  • $\begingroup$ Yes, it is an unspoken rule that the sample size is known in the background. That has the consequence that in many simple examples each of the sum and the mean of the observations is sufficient on its own, such as in your Bernoulli example $\endgroup$
    – Henry
    Jun 28, 2022 at 18:19
  • $\begingroup$ An intuition that is known comes even from the notation of the statistic where you sum from 1 to $n$ $\endgroup$
    – rapaio
    Jun 28, 2022 at 18:25
  • 2
    $\begingroup$ Do you also need to know the stopping rules? Data from negative sampling (i.e. sampling until a pre-determined number of successes is observed) has a different frequentist interpretation from the same results from normal sampling (where the total number of tries is fixed in advance). $\endgroup$ Jun 28, 2022 at 21:13
  • 1
    $\begingroup$ Following @MichaelLew $n$ is part of the sufficient statistic only when $n$ is the outcome of a random variable $N$, for which a distribution must be chosen/provided. Otherwise, if $n$ is fixed, it is irrelevant to include it in a statistic. $\endgroup$
    – Xi'an
    Jun 29, 2022 at 7:35

1 Answer 1

2
$\begingroup$

It is typical practice that the sample size is considered (implicitly) to be a known constant unless we specify the contrary in the analysis. This practice saves time by alleviating the need to specify that the sample size is known, which is true in the vast majority of statistical applications. You can of course proceed on the basis that $n$ is also an unknown parameter in the model. In this latter case your log-likelihood function would be:

$$\ell_{\mathbf{x}_n}(n,\theta) = \log {n \choose T(\mathbf{x}_n)} + T(\mathbf{x}_n) \log(\theta) + (n-T(\mathbf{x}_n)) \log(1-\theta),$$

and the minimal sufficient statistic is indeed $(n,T(\mathbf{x}_n))$ (so the statistic $T(\mathbf{x}_n)$ is not sufficient in this case).

(Note: I do not agree with the comment by Xi'an asserting that $n$ must be the outcome of a random variable to be included as part of the sufficient statistic; the concept of sufficiency is a classical concept, and in that domain the notion of an "unknown constant" is perfectly valid. There is no need to create a Bayesian model that specifies a distribution for $n$ in order for it to be part of the sufficient statistic.)

$\endgroup$
1
  • $\begingroup$ +1 for the parenthetical remark: sufficiency pertains to data reduction. Unless otherwise specified, $n$ is an observable quantity reflecting the amount of data that is observed. $\endgroup$
    – heropup
    Jul 5, 2022 at 1:58

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.