# Sufficient statistics

I have independent observations $Y_1, Y_2, ...., Y_n$ that take values 1 or 0 with

$$P(Y_i = 1) = \frac{1}{1 + \exp\{-(\alpha + \beta x_i)\}}$$

where $\alpha$ and $\beta$ are unknown constant and $x_1, x_2, ....,x_n$ are known, I have to show that $(\sum Y_i,\sum x_i Y_i)$is a minimal sufficient statistics for $(\alpha,\beta)$.

We know that any statistics $T(Y)$ is minimal sufficient if $\frac{\prod_{i=1}^nf(y_i;\alpha, \beta)}{\prod_{i=1}^nf(z_i;\alpha, \beta)}$ is independent of $\alpha$ and $\beta$ if and only if $T(Y) = T(Z)$. But here the probability density of $Y$ is independent of $y$. Is there any other method to show the minimal statistics?

-
The probability density of $Y$ is not independent from $y$! –  Xi'an Mar 16 '12 at 9:43
As @Xi'an has mentioned the density of $Y$ is not independent from $y$. You can see this by using inidcator function, for example you can write $P(Y_i = 1)=\frac{1}{1+\exp\{-(\alpha + \beta x_i)\}}I_{1}(y_i)$, in which $I_{1}(y_i)=1$ when $y_i=1$ and is zero when $y_i=0$. –  hbaghishani Mar 16 '12 at 11:02

Write $Y=(Y_1,\dots,Y_n)$ and $y=(y_1,\dots,y_n)$. Use the notation $$p(y_i\mid\alpha,\beta)=P\{Y_i=y_i\mid\alpha,\beta\}\, .$$

Your initial difficulty seems to be related to the fact that you fail to see that $$Y_i\mid\alpha,\beta\sim \mathrm{Ber}\left(\frac{1}{1+e^{-(\alpha+\beta x_i)}}\right) \, .$$

Now, you have to remember that for any Bernoulli random variable you can write its probability function as $$\left( \textrm{pr. of success} \right)^\textrm{value of the r.v.} \left( \textrm{pr. of failure} \right)^\textrm{1 - value of the r.v.} \, .$$

So, in the present case we have $$p(y_i\mid\alpha,\beta) = \left( \frac{1}{1+e^{-(\alpha+\beta x_i)}} \right)^{y_i} \left( \frac{e^{-(\alpha+\beta x_i)}}{1+e^{-(\alpha+\beta x_i)}}\right)^{1-y_i} \, .$$

Now, using conditional independence as usual, you will find (by simple algebra that I will not provide) that $$p(y\mid\alpha,\beta) = \prod_{i=1}^n p(y_i\mid\alpha,\beta) = \frac{e^{-\left( n\alpha +\beta \sum_{i=1}^n x_i -\alpha\sum_{i=1}^n y_i -\beta \sum_{i=1}^n x_i y_i \right)}}{\prod_{i=1}^n \left( 1+e^{-(\alpha+\beta x_i)} \right) } \, ,$$ and from that (why?) you can infer that $T(Y)=\left( \sum_{i=1}^n Y_i,\sum_{i=1}^n x_i Y_i\right)$ is sufficient for $(\alpha,\beta)$.

As for the minimality, from the above results, it is easy to see that $$\frac{p(y\mid\alpha,\beta)}{p(z\mid\alpha,\beta)} = \exp \left( \alpha\left(\sum_{i=1}^n y_i - \sum_{i=1}^n z_i \right) + \beta\left(\sum_{i=1}^n x_i y_i - \sum_{i=1}^n x_i z_i \right) \right) \, ,$$ and this ratio, seem as a function of $(\alpha,\beta)$, is a constant if and only if $\sum_{i=1}^n y_i = \sum_{i=1}^n z_i$ and $\sum_{i=1}^n x_i y_i = \sum_{i=1}^n x_i z_i$ (why?), which is equivalent to $T(y)=T(z)$.

-

I think this is a homework question. If so, you should tag it as such. Under the assumption that this is homework, I'm just going to give you enough to get you started.

We have

$P(Y_i = 1) = \frac{1}{1 + \exp\{-(\alpha + \beta x_i)\}}$

so it must be the case that

$P(Y_i = 0) = 1 - \frac{1}{1 + \exp\{-(\alpha + \beta x_i)\}} = \frac{\exp\{-(\alpha + \beta x_i)\}}{1 + \exp\{-(\alpha + \beta x_i)\}}$.

To formulate this in a way that relates the random variable $Y_i$ and the outcome $y_i$, we can write

$P(Y_i = y_i) = \frac{\exp\{-(1 - y_i)(\alpha + \beta x_i)\}}{1 + \exp\{-(\alpha + \beta x_i)\}}$.

You can verify by substitution that this gives the correct expressions for $\Pr(Y_i = 1)$ and $\Pr(Y_i = 0)$.

-
Why not give another hint such as $\exp(c)\times\exp(d)= \exp(c+d)$? –  Henry Mar 16 '12 at 1:33
Thank you so much. –  David Mar 16 '12 at 2:33
If this answer solves your problem, I'd appreciate it if you could click the little checkmark next to it -- I have a question I want to put a bounty on, so I have a use for the reputation points it will give me. –  Cyan Mar 16 '12 at 4:35