MLE for the weights of multilayer perceptron assuming a negative binomial distribution I am fitting a multilayer percepton with one hidden layer and $4$ neurons in this layer for regression. I need to calculate the likelihood for the outcomes because I need this for the BIC. All the literature I found assumes that $y \sim N(mlp(x_i,w), \sigma^2)$ but my outcome is not normally distributed. It rather follows a poisson or negative binomial distribution. How do I calculate the MLE for the weights then? 
If $y$ was normally distributed, it is easy because by minimizing $E(w) = \sum (y_i - mlp(x_i,w))^2$ with some stochastic gradient method I get the MLE. But I do not know how to find the MLE when I assume a non-normal distribution.
 A: MLE is a general framework, the same approach could be applied to other distributions. It's not that difficult to find function that you need to minimize with gradient descent. For example, MLE for poisson: https://en.wikipedia.org/wiki/Poisson_regression#Maximum_likelihood-based_parameter_estimation
It's a bit more difficult for the negative binomial distribution because it has two parameters. In the gaussian distribution, there are also two parameters, but one of them is fixed (variance). For the negative binomial, you also need to fix one of the parameters. If you're interested in integer predictions then you need to fix parameter that corresponds to the probability of success. You can use similar approach in order to find log likelihood for the function that you need to optimize: https://en.wikipedia.org/wiki/Negative_binomial_distribution#Maximum_likelihood_estimation
In general, both of these approaches follow the same procedure. First, you find log-likelihood for your estimations and then you apply gradient descent on the negative log-likelihood with respect to the parameter. Basically, you just create different loss functions for the neural network training. Of course, you need to use all your training samples in order to obtain the solution for the MLE.
A: Two key points:


*

*You can apply MLE on the likelihood function of any distribution:


$~~~~~~~\langle \hat w_1,...,\hat w_n \rangle = \text{arg}\,\max\limits_{w_1,...,w_n}\ \mathcal{L}(w_1,...,w_n \mid Data ) = \text{arg}\,\max\limits_{w_1,...,w_n}\ \mathcal{f}(Data \mid w_1,...,w_n  )$


*If your $y$ vector was subsampled from a larger dataset, you can always assume asymptotic normality: 


$~~~~~~~y~\dot\sim~ N(mlp(x_i,w), \sigma^2)$
