0
$\begingroup$

I have derived a likelihood function for $\theta$ as follows:

$$L(\theta)=(2\pi\theta)^{-n/2} \exp\left(\frac{ns}{2\theta}\right)$$

Where $\theta$ is an unknown parameter, $n$ is the sample size, and $s$ is a summary of the data. I now am trying to show that $s$ is a sufficient statistic for $\theta$.

In Wikipedia the Fischer-Neyman factorization is described as:

$$f_\theta(x)=h(x)g_\theta(T(x))$$

My first question is notation. In my problem I believe what wikipedia represents as $x$, is $\theta$, and what wikipedia represents as $\theta$ is $s$. Please confirm that that sounds right, it's a point of confusion for me.

Which would mean I'm trying to define the following 3 functions to complete the factorization and confirm that $s$ is sufficient for $\theta$

$$T(\theta)$$ $$g_s(T(\theta))$$ $$h(\theta)$$

But by this point I feel like I've done something wrong, and I'm not really understanding why this factorization is demonstrating sufficiency. I don't really see what is going on with $g$ and $T$.

$\endgroup$
  • $\begingroup$ $x$ is the data and $T(x)$ is the sufficient statistic, or $s$ in your usage, which is a function of the data and only of the data. The idea is that if the likelihood function can be entirely written in terms of $T(x)$ instead of in terms of the individual $x_i, i=1, \dots, N$, then $T(x)$ is a sufficient statistic (which may be multidimensional, for example, in the case of the Normal distribution.) $\endgroup$ – jbowman Mar 7 '18 at 1:30
  • $\begingroup$ So then the likelihood function is already in a factorized form with $h(x)=1$? In this case $s = \sum{x_i^2}$ where each $x_i$ is a sample from the dataset. I'm eternally suspicious when something seems too easy to be true. $\endgroup$ – David Parks Mar 7 '18 at 2:09
  • $\begingroup$ Yes, that's correct, but then they do pick the easy examples for working through / learning from. Incidentally, it doesn't matter what $h(x)$ is when you are trying to find the maximum likelihood estimator (or any number of other things), it's just a constant that has no effect on the shape of the likelihood function itself. $\endgroup$ – jbowman Mar 7 '18 at 2:38
  • $\begingroup$ Thanks @jbowman, that was very helpful, please do post that as an answer, that's helped me understand what I was missing. $\endgroup$ – David Parks Mar 7 '18 at 2:44
  • $\begingroup$ Add self study tag. $\endgroup$ – Michael Chernick Mar 7 '18 at 3:17
1
$\begingroup$

In this case, $x$ is the data and $T(x)$ is the sufficient statistic. In your initial expression, $T(x)$ is represented by $ns = \sum_i x_i$. In a (more canonical) representation of the likelihood function we could write:

$$L(\theta)=(2\pi\theta)^{-n/2} \exp\left(\frac{T(x)}{2\theta}\right)$$

In this case, $h(x)=1$ and $g_{\theta}(T(x)) = (2\pi\theta)^{-n/2} \exp(T(x)/2\theta)$, i.e., the entire likelihood function - although we can drop the constant term $(2\pi)^{-n/2}$ without any ill-effect.

The idea is that if the likelihood function can be entirely written in terms of $T(x)$ instead of in terms of the individual $x_i,i=1,…,N$, then $T(x)$ is a sufficient statistic (which may be multidimensional, for example, in the case of the Normal distribution.) In this case the only value that counts in terms of the likelihood function is the sum of the $x_i$, not the individual $x_i$ values themselves, so $\sum_i x_i$ is a sufficient statistic.

It doesn't matter what $h(x)$ is from a likelihood point of view, as the likelihood is a function of the parameters $\theta$; $h(x)$ is just a constant that has no effect on the shape of the likelihood function itself.

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.