Using a Neural Network as a replacement for Fractional Logit; Which Cost Function to Use? I am working on a deep neural net model to estimate the monthly mortgage prepayment ratio for a large loan portfolio (this ratio is called $SMM$ in finance literature, all you need to know is that $0\leq SMM\leq1$). My basic approach is to extend the methodology in (Papke- Wooldridge 1996) paper on modelling fractional outcomes. So I started by


*

*replacing the $G(z)$ (sigmoid function in the above paper) by a deep NN.

*minimizing the negative Bernoulli log-likelihood function:
$$\text{argmin}_\theta\left(-\sum_i y_i\log[G(X_i\theta)]+(1-y_i)\log[1-G(X_i\theta)]\right)$$
My question is:
what is the conceptual difference between using the above cost function as opposed to the usual Mean Squared Error cost for a regular quantitative response $0\leq y \leq1$?
 A: Ultimately they are just two different ways of estimating the mean, $E_{x_i}[y_i]$ using two different methods. With square error loss, you are assuming that your training set $$y_i=f(x_i;\theta)+\epsilon$$ where $\epsilon\sim\mathcal{N}(0,\sigma^2)$ as the mean squared error is equivalent to the scaled and shifted Gaussian negative log-likelihood and therefore has the same minimizer. Therefore it is an exact MLE estimator when the variance is (approximately) constant. 
With your NN case, you are using a Bernoulli pdf-based quasi-likelihood estimator for the same expectation which is asymptotically accurate if $f$ is a viable estimator for all $(x_i,y_i)$. It is a quasi-likelihood because we cannot assume the true pdf is in this particular family and also because this likelihood is missing a normalization function related to $f$ and $\theta$. This estimator has some nice properties of being consistent regardless of the distribution of $y_i$ as compared to typical GLM with logit link function. However, the mean squared error estimator is  similarly consistent given that both distributions are part of the linear exponential family. 
Furthermore, $f$ can be bound to $[0,1]$ in either model, and the mean squared error estimator minimizes the prediction discrepancy directly and is computationally simplier. 
The paper you cite and the paper it references for it's main results suggest some differences in efficiency if the variance is closer to normal or Bernoulli ($v=\hat{y_i}(1-\hat{y_i})$), but for a very large data set, this may not change the accuracy of your predictions much.
