I have independent observations $Y_1, Y_2, ...., Y_n$ that take values 1 or 0 with

$$P(Y_i = 1) = \frac{1}{1 + \exp\{-(\alpha + \beta x_i)\}}$$

where $\alpha$ and $\beta$ are unknown constant and $x_1, x_2, ....,x_n$ are known, I have to show that $(\sum Y_i,\sum x_i Y_i)$is a minimal sufficient statistics for $(\alpha,\beta)$.

We know that any statistics $T(Y)$ is minimal sufficient if $\frac{\prod_{i=1}^nf(y_i;\alpha, \beta)}{\prod_{i=1}^nf(z_i;\alpha, \beta)}$ is independent of $\alpha$ and $\beta$ if and only if $T(Y) = T(Z)$. But here the probability density of $Y$ is independent of $y$. Is there any other method to show the minimal statistics?

  • 3
    $\begingroup$ The probability density of $Y$ is not independent from $y$! $\endgroup$
    – Xi'an
    Mar 16, 2012 at 9:43
  • $\begingroup$ As @Xi'an has mentioned the density of $Y$ is not independent from $y$. You can see this by using inidcator function, for example you can write $P(Y_i = 1)=\frac{1}{1+\exp\{-(\alpha + \beta x_i)\}}I_{1}(y_i)$, in which $I_{1}(y_i)=1$ when $y_i=1$ and is zero when $y_i=0$. $\endgroup$ Mar 16, 2012 at 11:02

2 Answers 2


Write $Y=(Y_1,\dots,Y_n)$ and $y=(y_1,\dots,y_n)$. Use the notation $$p(y_i\mid\alpha,\beta)=P\{Y_i=y_i\mid\alpha,\beta\}\, .$$

Your initial difficulty seems to be related to the fact that you fail to see that $$Y_i\mid\alpha,\beta\sim \mathrm{Ber}\left(\frac{1}{1+e^{-(\alpha+\beta x_i)}}\right) \, .$$

Now, you have to remember that for any Bernoulli random variable you can write its probability function as $$ \left( \textrm{pr. of success} \right)^\textrm{value of the r.v.} \left( \textrm{pr. of failure} \right)^\textrm{1 - value of the r.v.} \, . $$

So, in the present case we have $$ p(y_i\mid\alpha,\beta) = \left( \frac{1}{1+e^{-(\alpha+\beta x_i)}} \right)^{y_i} \left( \frac{e^{-(\alpha+\beta x_i)}}{1+e^{-(\alpha+\beta x_i)}}\right)^{1-y_i} \, . $$

Now, using conditional independence as usual, you will find (by simple algebra that I will not provide) that $$ p(y\mid\alpha,\beta) = \prod_{i=1}^n p(y_i\mid\alpha,\beta) = \frac{e^{-\left( n\alpha +\beta \sum_{i=1}^n x_i -\alpha\sum_{i=1}^n y_i -\beta \sum_{i=1}^n x_i y_i \right)}}{\prod_{i=1}^n \left( 1+e^{-(\alpha+\beta x_i)} \right) } \, , $$ and from that (why?) you can infer that $T(Y)=\left( \sum_{i=1}^n Y_i,\sum_{i=1}^n x_i Y_i\right)$ is sufficient for $(\alpha,\beta)$.

As for the minimality, from the above results, it is easy to see that $$ \frac{p(y\mid\alpha,\beta)}{p(z\mid\alpha,\beta)} = \exp \left( \alpha\left(\sum_{i=1}^n y_i - \sum_{i=1}^n z_i \right) + \beta\left(\sum_{i=1}^n x_i y_i - \sum_{i=1}^n x_i z_i \right) \right) \, , $$ and this ratio, seem as a function of $(\alpha,\beta)$, is a constant if and only if $\sum_{i=1}^n y_i = \sum_{i=1}^n z_i$ and $\sum_{i=1}^n x_i y_i = \sum_{i=1}^n x_i z_i$ (why?), which is equivalent to $T(y)=T(z)$.


I think this is a homework question. If so, you should tag it as such. Under the assumption that this is homework, I'm just going to give you enough to get you started.

We have

$P(Y_i = 1) = \frac{1}{1 + \exp\{-(\alpha + \beta x_i)\}}$

so it must be the case that

$P(Y_i = 0) = 1 - \frac{1}{1 + \exp\{-(\alpha + \beta x_i)\}} = \frac{\exp\{-(\alpha + \beta x_i)\}}{1 + \exp\{-(\alpha + \beta x_i)\}}$.

To formulate this in a way that relates the random variable $Y_i$ and the outcome $y_i$, we can write

$P(Y_i = y_i) = \frac{\exp\{-(1 - y_i)(\alpha + \beta x_i)\}}{1 + \exp\{-(\alpha + \beta x_i)\}}$.

You can verify by substitution that this gives the correct expressions for $\Pr(Y_i = 1)$ and $\Pr(Y_i = 0)$.

  • $\begingroup$ Why not give another hint such as $\exp(c)\times\exp(d)= \exp(c+d)$? $\endgroup$
    – Henry
    Mar 16, 2012 at 1:33
  • 4
    $\begingroup$ If this answer solves your problem, I'd appreciate it if you could click the little checkmark next to it -- I have a question I want to put a bounty on, so I have a use for the reputation points it will give me. $\endgroup$
    – Cyan
    Mar 16, 2012 at 4:35

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge that you have read and understand our privacy policy and code of conduct.

Not the answer you're looking for? Browse other questions tagged or ask your own question.