Sufficient statistics

Question

I have independent observations $Y_1, Y_2, ...., Y_n$ that take values 1 or 0 with

$$P(Y_i = 1) = \frac{1}{1 + \exp\{-(\alpha + \beta x_i)\}}$$

where $\alpha$ and $\beta$ are unknown constant and $x_1, x_2, ....,x_n$ are known, I have to show that $(\sum Y_i,\sum x_i Y_i)$is a minimal sufficient statistics for $(\alpha,\beta)$.

We know that any statistics $T(Y)$ is minimal sufficient if $\frac{\prod_{i=1}^nf(y_i;\alpha, \beta)}{\prod_{i=1}^nf(z_i;\alpha, \beta)}$ is independent of $\alpha$ and $\beta$ if and only if $T(Y) = T(Z)$. But here the probability density of $Y$ is independent of $y$. Is there any other method to show the minimal statistics?

Answer 1

Write $Y=(Y_1,\dots,Y_n)$ and $y=(y_1,\dots,y_n)$. Use the notation $$p(y_i\mid\alpha,\beta)=P\{Y_i=y_i\mid\alpha,\beta\}\, .$$

Your initial difficulty seems to be related to the fact that you fail to see that $$Y_i\mid\alpha,\beta\sim \mathrm{Ber}\left(\frac{1}{1+e^{-(\alpha+\beta x_i)}}\right) \, .$$

Now, you have to remember that for any Bernoulli random variable you can write its probability function as $$ \left( \textrm{pr. of success} \right)^\textrm{value of the r.v.} \left( \textrm{pr. of failure} \right)^\textrm{1 - value of the r.v.} \, . $$

So, in the present case we have $$ p(y_i\mid\alpha,\beta) = \left( \frac{1}{1+e^{-(\alpha+\beta x_i)}} \right)^{y_i} \left( \frac{e^{-(\alpha+\beta x_i)}}{1+e^{-(\alpha+\beta x_i)}}\right)^{1-y_i} \, . $$

Now, using conditional independence as usual, you will find (by simple algebra that I will not provide) that $$ p(y\mid\alpha,\beta) = \prod_{i=1}^n p(y_i\mid\alpha,\beta) = \frac{e^{-\left( n\alpha +\beta \sum_{i=1}^n x_i -\alpha\sum_{i=1}^n y_i -\beta \sum_{i=1}^n x_i y_i \right)}}{\prod_{i=1}^n \left( 1+e^{-(\alpha+\beta x_i)} \right) } \, , $$ and from that (why?) you can infer that $T(Y)=\left( \sum_{i=1}^n Y_i,\sum_{i=1}^n x_i Y_i\right)$ is sufficient for $(\alpha,\beta)$.

As for the minimality, from the above results, it is easy to see that $$ \frac{p(y\mid\alpha,\beta)}{p(z\mid\alpha,\beta)} = \exp \left( \alpha\left(\sum_{i=1}^n y_i - \sum_{i=1}^n z_i \right) + \beta\left(\sum_{i=1}^n x_i y_i - \sum_{i=1}^n x_i z_i \right) \right) \, , $$ and this ratio, seem as a function of $(\alpha,\beta)$, is a constant if and only if $\sum_{i=1}^n y_i = \sum_{i=1}^n z_i$ and $\sum_{i=1}^n x_i y_i = \sum_{i=1}^n x_i z_i$ (why?), which is equivalent to $T(y)=T(z)$.

Answer 2

I think this is a homework question. If so, you should tag it as such. Under the assumption that this is homework, I'm just going to give you enough to get you started.

We have

$P(Y_i = 1) = \frac{1}{1 + \exp\{-(\alpha + \beta x_i)\}}$

so it must be the case that

$P(Y_i = 0) = 1 - \frac{1}{1 + \exp\{-(\alpha + \beta x_i)\}} = \frac{\exp\{-(\alpha + \beta x_i)\}}{1 + \exp\{-(\alpha + \beta x_i)\}}$.

To formulate this in a way that relates the random variable $Y_i$ and the outcome $y_i$, we can write

$P(Y_i = y_i) = \frac{\exp\{-(1 - y_i)(\alpha + \beta x_i)\}}{1 + \exp\{-(\alpha + \beta x_i)\}}$.

You can verify by substitution that this gives the correct expressions for $\Pr(Y_i = 1)$ and $\Pr(Y_i = 0)$.