2
$\begingroup$

One can set the parameter “stopping_metric” to “deviance” for the deeplearning algorithm (http://docs.h2o.ai/h2o/latest-stable/h2o-py/docs/modeling.html#h2o.estimators.deeplearning.H2ODeepLearningEstimator.stopping_metric).

In the FAQs (http://docs.h2o.ai/h2o/latest-stable/h2o-docs/data-science/deep-learning.html), the answer to the question “How is deviance computed for a Deep Learning regression model?” is given by the following formula:

Loss = Quadratic -> MSE==Deviance For Absolute/Laplace or Huber -> MSE != Deviance.

As far as I understand, the deviance equals the MSE if the loss function is quadratic. How is it defined in the other cases (e.g. Huber)? I read the definition that in this case the deviance is not the MSE but what is it then?

Looking forward to your answer. Thanks in advance.

$\endgroup$
0
$\begingroup$

The code seems to be here:

https://github.com/h2oai/h2o-3/blob/5403e7c7d1e787013105ca977c1a62c249b8ed61/h2o-core/src/main/java/hex/Distribution.java#L69

  case huber:
    if (Math.abs(y-f) <= huberDelta) {
      return w * (y - f) * (y - f); // same as wMSE
    } else {
      return 2 * w * (Math.abs(y-f) - huberDelta)*huberDelta; // note quite the same as wMAE
    }

I.e. if the error is less than huberDelta then it is MSE, otherwise it is a kind of MAE.

E.g. if y (correct value) is 0.5, f (the prediction) is 0.7, and huberDelta is 0.05,(and w is 1.0, for simplicity), MSE would be 0.04, but Huber deviance will be 2 * 0.15 * 0.05 = 0.015. If f was 0.6, MSE would be 0.01, while Huber deviance would be 2 * 0.05 * 0.05 = 0.005.

The huberDelta is set here: https://github.com/h2oai/h2o-3/blob/5403e7c7d1e787013105ca977c1a62c249b8ed61/h2o-algos/src/main/java/hex/deeplearning/DeepLearningModel.java#L387

double huberDelta = MathUtils.computeWeightedQuantile(fTrain.vec(get_params()._weights_column), absdiff, get_params()._huber_alpha);

Huber Alpha, in turn, can be between 0.0 and 1.0, but defaults to 0.9: http://docs.h2o.ai/h2o/latest-stable/h2o-docs/data-science/algo-params/huber_alpha.html

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.