4
$\begingroup$

does anyone know how exactly the Deviances (Poisson, Gamma, Tweedie) are computed in H2O? I cannot find the functions. For interpretation purposes I would like to know the calculations.

Thank you!

$\endgroup$
2
+50
$\begingroup$

The formulas are in the H2O User Guide in the GLM section. Here are links for the deviance formulas for

  1. Linear: $$D=\sum^{N}_{i=1}{(y_{i}-\hat{y}_{i})^{2}}$$
  2. Logistic: $$-D=\sum^{N}_{i=1}{\left(y_{i}\log{(\hat{y}_{i})}+(1-y_{i})\log{(1-\hat{y}_{i})}\right)}$$
  3. Poisson: $$D=-2\sum^{N}_{i=1}{\left(y_{i}\log{\left(\frac{y_{i}}{\hat{y}_{i}}\right)}-\left(y_{i}-\hat{y}_{i}\right)\right)}$$
  4. Gamma: $$D=2\sum^{N}_{i=1}{\left(-\log{\left(\frac{y_{i}}{\hat{y}_{i}}\right)}+\frac{\left(y_{i}-\hat{y}_{i}\right)}{\hat{y}_{i}}\right)}$$
  5. Tweedie: where $p\neq1$ and $p\neq2$ $$D=-2\sum^{N}_{i=1}{\left(\frac{y_{i}\left(y_{i}^{1-p}-\hat{y}_{i}^{1-p}\right)}{1-p}-\frac{\left(y_{i}^{2-p}-\hat{y}_{i}^{2-p}\right)}{2-p}\right)}$$

Note for Tweedie there is an error in the docs, you can see the formula actually used in the h2o source, line 395 here: https://github.com/h2oai/h2o-3/blob/master/h2o-algos/src/main/java/hex/glm/GLMModel.java. The formula in this answer shows the correct formula rather than the formula in the docs.

| cite | improve this answer | |
$\endgroup$
0
$\begingroup$

The deviance equations in H2O should be following the general specifications used typically in actuarial sciences. See for instance Modern Actuarial Risk Theory by Kaas et al. page $308$ for the Tweedie Deviance and pages $246$ and $247$ for Poisson and Gamma Deviances. They correspond to the same equations in the accepted answer above, save the weights.

| cite | improve this answer | |
$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.