# using DNN to find out the pdf of a regression problem

When we use deep neural networks (DNNs) to solve a 1-dimention regression problem, we can approximate data distribution with the output of a DNN like the picture below.
My question is that DNN does not have the assumption of gaussian distribution or any other distribution of itself. It just knows what value to output when it sees an input. So how do you know the probability distribution of the DNN? For example, if someone asks, what is the probability of the point appearing in (5, 0). Can DNN answer this kind of questions?

For many regression algorithms, not only neural networks, the model is that the data is distributed by $y \sim \mathcal{N}(f(x;\theta), \sigma^2)$, where $\theta$ are the model parameters and $\sigma^2$ is the variance of the distribution (often a hyperparameter).
Maximizing the log-likelihood of the data with respect to $\theta$ is equivalent to minimizing the mean squared error loss between the $y_i$ and $f(x_i;\theta)$.
Therefore, to compute the probability density of $(5,0)$, you would just find the density of a gaussian with mean $f(5; \theta)$ and a variance of $\sigma^2$, where $f$ is your neural network.
• Yes, it still applies. The only assumption we need to apply MSE loss and obtain our PDF is that the distribution of the target $y$ is a function of the input $x$ plus some gaussian noise. That function $f$ doesn't have to be linear and in this case is modeled by a highly nonlinear neural network. – shimao Feb 2 '18 at 2:30