I know that neural nets use activation functions, but where do distribution functions play into deep neural networks? For example, the h2o.deeplearning() function in R has the variable distribution = c("AUTO", "gaussian", "bernoulli", "multinomial", "poisson", "gamma", "tweedie", "laplace", "huber", "quantile"). Where does this apply in deep learning? Also, I came across this quote from an h2o tutorial that says "H2O Deep Learning supports regression for distributions other than Gaussian such as Poisson, Gamma, Tweedie, Laplace", but again I am confused as to where a distribution function plays into the concept of multilayer perceptrons.

  • 2
    $\begingroup$ Currently the question is very intimately related to particular software; as such it seems to be off topic. But you could try making it less software specific, since I think you do have a genuine statistical/machine learning question here. $\endgroup$ – Richard Hardy Jul 11 '16 at 10:17
  • $\begingroup$ IMO, this is enough machine learning content to be on topic here. The question doesn't seem to really be about how to use the R functions. $\endgroup$ – gung - Reinstate Monica Jul 11 '16 at 12:51

I haven't used H2O yet, but a quick look at the documentation shows that the distribution parameter concerns with regression models (more precisely generalized linear models), e.g.

... H2O Deep Learning was run with distribution=AUTO, which defaulted to a Gaussian regression problem for a real-valued response. H2O Deep Learning supports regression for distributions other than Gaussian such as Poisson, Gamma, Tweedie, Laplace.

The same is mentioned in those slides. In regression problem you have vector of predicted values $Y$ and a matrix of predictors $\boldsymbol{X}$ and we want to learn about parameters $\boldsymbol{\beta}$. For this, we estimate linear predictor

$$ \eta = \boldsymbol{X\beta} $$

that is used along with appropriate link function $g$ such that

$$ E(Y) = \mu = g^{-1}(\eta) $$

where $\mu$ is mean of some distribution, e.g. normal to be used for linear regression, Laplace to more robust version of linear regression, Bernoulli or binomial for logistic regression, with other possibilities being gamma, lognormal, negative binomial, Poisson and other distributions.

To read more about GLM's you can check the book by McCullagh and Nelder (1989).

| cite | improve this answer | |

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.