I found the deviance definition in https://en.wikipedia.org/wiki/Deviance_(statistics) and the one-observation bernoulli deviance in Scikit Binomial Deviance Loss Function. $$ \text{bernoulliDeviance}_i = -2 * (y_i\log(p_i)+(1-y_i)\log(1-p_i)). $$ How can I derive such formula by the wiki definition of deviance?

  • $\begingroup$ Your formula is so close to the one in the Wikipedia article that we are left to wonder what additional help you are looking for. Could you articulate the nature of the obstacle that is preventing you from connecting the two? $\endgroup$
    – whuber
    Apr 20, 2016 at 13:00
  • 2
    $\begingroup$ how to calculate the saturated model log(p(y | theta_s)) $\endgroup$
    – ScutterKey
    Apr 20, 2016 at 14:38

1 Answer 1



  1. $p \equiv$ your $p_i$, the bernoulli parameter, such that $P(1) = p$

  2. $\hat{\theta}_0 \equiv p$, the general parameter vector is, in this case, equal to the bernoulli parameter

With that, we can write the function with the two possible states:

  • if $y_i = 0$, $p(y|p) = (1-p)$
  • if $y_i = 1$, $p(y|p) = p$

We can combine those using something akin to a delta function by attaching a prefactor that is $1$ when for the corresponding state of $y_i$:

$f(y_i) = (1-y_i) log(1-p) + y_i log(p)$

You can check that this function gives the same values as the conditional function at 0 and 1.

  • $\begingroup$ I can not figure out what is the saturated model log(p(y | theta_s)). $\endgroup$
    – ScutterKey
    Sep 23, 2016 at 15:03

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge that you have read and understand our privacy policy and code of conduct.

Not the answer you're looking for? Browse other questions tagged or ask your own question.