How do distribution functions (e.g. Gaussian, Bernoulli, Poisson, etc.) relate to deep learning? I know that neural nets use activation functions, but where do distribution functions play into deep neural networks? For example, the h2o.deeplearning() function in R has the variable distribution = c("AUTO", "gaussian", "bernoulli", "multinomial", "poisson", "gamma", "tweedie", "laplace", "huber", "quantile").  Where does this apply in deep learning? Also, I came across this quote from an h2o tutorial that says "H2O Deep Learning supports regression for distributions other than Gaussian such as Poisson, Gamma, Tweedie, Laplace", but again I am confused as to where a distribution function plays into the concept of multilayer perceptrons.
 A: I haven't used H2O yet, but a quick look at the documentation shows that the distribution parameter concerns with regression models (more precisely generalized linear models), e.g.

... H2O Deep Learning was run with distribution=AUTO, which defaulted
  to a Gaussian regression problem for a real-valued response. H2O Deep
  Learning supports regression for distributions other than Gaussian
  such as Poisson, Gamma, Tweedie, Laplace.

The same is mentioned in those slides. In regression problem you have vector of predicted values $Y$ and a matrix of predictors $\boldsymbol{X}$ and we want to learn about parameters $\boldsymbol{\beta}$. For this, we estimate linear predictor
$$ \eta = \boldsymbol{X\beta} $$
that is used along with appropriate link function $g$ such that
$$ E(Y) = \mu = g^{-1}(\eta) $$
where $\mu$ is mean of some distribution, e.g. normal to be used for linear regression, Laplace to more robust version of linear regression, Bernoulli or binomial for logistic regression, with other possibilities being gamma, lognormal, negative binomial, Poisson and other distributions.
To read more about GLM's you can check the book by McCullagh and Nelder (1989).
