11
$\begingroup$

From some machine learning notes talking about some discriminative classification methods, in particular logistic regression, where y is the class label (0 or 1) and x is the data, it is said that:

if $x|y = 0 \sim \mathrm{Poisson}(λ_0)$, and $x|y = 1 \sim \mathrm{Poisson}(λ_1)$, then $p(y|x)$ will be logistic.

Why is this true?

$\endgroup$

1 Answer 1

16
$\begingroup$

$Y$ has two possible values for any given value of $X$. According to the assumptions,

$$\Pr(X=x|Y=0) = \exp(-\lambda_0) \frac{\lambda_0^x}{x!}$$

and

$$\Pr(X=x|Y=1) = \exp(-\lambda_1) \frac{\lambda_1^x}{x!}.$$

Therefore (this is a trivial case of Bayes' Theorem) the chance that $Y=1$ conditional on $X=x$ is the relative probability of the latter, namely

$$\Pr(Y=1|X=x) = \frac{\exp(-\lambda_1) \frac{\lambda_1^x}{x!}}{\exp(-\lambda_1) \frac{\lambda_1^x}{x!} + \exp(-\lambda_0) \frac{\lambda_0^x}{x!}}= \frac{1}{1 + \exp(\beta_0 + \beta_1 x)}$$

where

$$\beta_0 = \lambda_1 - \lambda_0$$

and

$$\beta_1 = -\log(\lambda_1/\lambda_0).$$

That indeed is the standard logistic regression model.

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.