Take the 2-minute tour ×
Cross Validated is a question and answer site for people interested in statistics, machine learning, data analysis, data mining, and data visualization. It's 100% free, no registration required.

I have independent observations $Y_1, Y_2, ...., Y_n$ that take values 1 or 0 with

$$P(Y_i = 1) = \frac{1}{1 + \exp\{-(\alpha + \beta x_i)\}}$$

where $\alpha$ and $\beta$ are unknown constant and $x_1, x_2, ....,x_n$ are known, I have to show that $(\sum Y_i,\sum x_i Y_i)$is a minimal sufficient statistics for $(\alpha,\beta)$.

We know that any statistics $T(Y)$ is minimal sufficient if $\frac{\prod_{i=1}^nf(y_i;\alpha, \beta)}{\prod_{i=1}^nf(z_i;\alpha, \beta)}$ is independent of $\alpha$ and $\beta$ if and only if $T(Y) = T(Z)$. But here the probability density of $Y$ is independent of $y$. Is there any other method to show the minimal statistics?

share|improve this question
2  
The probability density of $Y$ is not independent from $y$! –  Xi'an Mar 16 '12 at 9:43
    
As @Xi'an has mentioned the density of $Y$ is not independent from $y$. You can see this by using inidcator function, for example you can write $P(Y_i = 1)=\frac{1}{1+\exp\{-(\alpha + \beta x_i)\}}I_{1}(y_i)$, in which $I_{1}(y_i)=1$ when $y_i=1$ and is zero when $y_i=0$. –  hbaghishani Mar 16 '12 at 11:02

2 Answers 2

up vote 3 down vote accepted

Write $Y=(Y_1,\dots,Y_n)$ and $y=(y_1,\dots,y_n)$. Use the notation $$p(y_i\mid\alpha,\beta)=P\{Y_i=y_i\mid\alpha,\beta\}\, .$$

Your initial difficulty seems to be related to the fact that you fail to see that $$Y_i\mid\alpha,\beta\sim \mathrm{Ber}\left(\frac{1}{1+e^{-(\alpha+\beta x_i)}}\right) \, .$$

Now, you have to remember that for any Bernoulli random variable you can write its probability function as $$ \left( \textrm{pr. of success} \right)^\textrm{value of the r.v.} \left( \textrm{pr. of failure} \right)^\textrm{1 - value of the r.v.} \, . $$

So, in the present case we have $$ p(y_i\mid\alpha,\beta) = \left( \frac{1}{1+e^{-(\alpha+\beta x_i)}} \right)^{y_i} \left( \frac{e^{-(\alpha+\beta x_i)}}{1+e^{-(\alpha+\beta x_i)}}\right)^{1-y_i} \, . $$

Now, using conditional independence as usual, you will find (by simple algebra that I will not provide) that $$ p(y\mid\alpha,\beta) = \prod_{i=1}^n p(y_i\mid\alpha,\beta) = \frac{e^{-\left( n\alpha +\beta \sum_{i=1}^n x_i -\alpha\sum_{i=1}^n y_i -\beta \sum_{i=1}^n x_i y_i \right)}}{\prod_{i=1}^n \left( 1+e^{-(\alpha+\beta x_i)} \right) } \, , $$ and from that (why?) you can infer that $T(Y)=\left( \sum_{i=1}^n Y_i,\sum_{i=1}^n x_i Y_i\right)$ is sufficient for $(\alpha,\beta)$.

As for the minimality, from the above results, it is easy to see that $$ \frac{p(y\mid\alpha,\beta)}{p(z\mid\alpha,\beta)} = \exp \left( \alpha\left(\sum_{i=1}^n y_i - \sum_{i=1}^n z_i \right) + \beta\left(\sum_{i=1}^n x_i y_i - \sum_{i=1}^n x_i z_i \right) \right) \, , $$ and this ratio, seem as a function of $(\alpha,\beta)$, is a constant if and only if $\sum_{i=1}^n y_i = \sum_{i=1}^n z_i$ and $\sum_{i=1}^n x_i y_i = \sum_{i=1}^n x_i z_i$ (why?), which is equivalent to $T(y)=T(z)$.

share|improve this answer

I think this is a homework question. If so, you should tag it as such. Under the assumption that this is homework, I'm just going to give you enough to get you started.

We have

$P(Y_i = 1) = \frac{1}{1 + \exp\{-(\alpha + \beta x_i)\}}$

so it must be the case that

$P(Y_i = 0) = 1 - \frac{1}{1 + \exp\{-(\alpha + \beta x_i)\}} = \frac{\exp\{-(\alpha + \beta x_i)\}}{1 + \exp\{-(\alpha + \beta x_i)\}}$.

To formulate this in a way that relates the random variable $Y_i$ and the outcome $y_i$, we can write

$P(Y_i = y_i) = \frac{\exp\{-(1 - y_i)(\alpha + \beta x_i)\}}{1 + \exp\{-(\alpha + \beta x_i)\}}$.

You can verify by substitution that this gives the correct expressions for $\Pr(Y_i = 1)$ and $\Pr(Y_i = 0)$.

share|improve this answer
    
Why not give another hint such as $\exp(c)\times\exp(d)= \exp(c+d)$? –  Henry Mar 16 '12 at 1:33
    
Thank you so much. –  David Mar 16 '12 at 2:33
4  
If this answer solves your problem, I'd appreciate it if you could click the little checkmark next to it -- I have a question I want to put a bounty on, so I have a use for the reputation points it will give me. –  Cyan Mar 16 '12 at 4:35

Your Answer

 
discard

By posting your answer, you agree to the privacy policy and terms of service.

Not the answer you're looking for? Browse other questions tagged or ask your own question.