# How to derive bernoulli deviance

I found the deviance definition in https://en.wikipedia.org/wiki/Deviance_(statistics) and the one-observation bernoulli deviance in Scikit Binomial Deviance Loss Function. $$\text{bernoulliDeviance}_i = -2 * (y_i\log(p_i)+(1-y_i)\log(1-p_i)).$$ How can I derive such formula by the wiki definition of deviance?

• Your formula is so close to the one in the Wikipedia article that we are left to wonder what additional help you are looking for. Could you articulate the nature of the obstacle that is preventing you from connecting the two?
– whuber
Apr 20, 2016 at 13:00
• how to calculate the saturated model log(p(y | theta_s)) Apr 20, 2016 at 14:38

Definitions:

1. $p \equiv$ your $p_i$, the bernoulli parameter, such that $P(1) = p$

2. $\hat{\theta}_0 \equiv p$, the general parameter vector is, in this case, equal to the bernoulli parameter

With that, we can write the function with the two possible states:

• if $y_i = 0$, $p(y|p) = (1-p)$
• if $y_i = 1$, $p(y|p) = p$

We can combine those using something akin to a delta function by attaching a prefactor that is $1$ when for the corresponding state of $y_i$:

$f(y_i) = (1-y_i) log(1-p) + y_i log(p)$

You can check that this function gives the same values as the conditional function at 0 and 1.

• I can not figure out what is the saturated model log(p(y | theta_s)). Sep 23, 2016 at 15:03