3
$\begingroup$

I'm wondering if it's okay to use the error function mean squared error (MSE) when using the multilayer perceptron for a regression problem where the target data is bimodal.

I standardize the inputs and log transform as well as standardize the target for faster convergence. But even when log transforming and afterwards standardizing the target the target data is not normal.

Is it wrong to use MSE as the error function if the target data is not normal or is there no such assumption for neural networks?

I can't think of any other error function that would be better to use if the assumptions would not be met, is there another error function that could be more appropriate in this case?

Help would be appreciated

$\endgroup$

2 Answers 2

3
$\begingroup$

Multimodal response is not a problem for MSE if that is due to a feature. You can get multimodality with a simple bivariate linear model. MSE assumes that data is conditionally normal , ie y given X is normal.

Using MSE becomes a problem when your data is bimodal (stochastic) conditioned on features too. Otherwise it's fine to use a normal model

Do not split data into clusters and fit 2 models later. That is unnecessary and defeats the purpose of a regression model in first place; y can have a thousand modes as long as y given X is Normal you are fine with a single model. Clustering is an inefficient use of data, unnecessary, and just adds noise (in most cases).

Better error function: If your data generating process is truly bimodal given features, then write down the log likelihood of a mixture model and use the negative log likelihood as loss. See also mixture density networks (MDN). But again in most cases normal distribution / MSE is good enough even with multimodal targets (anything else would be surprising in a real world regression setup anyways)

$\endgroup$
1
  • 1
    $\begingroup$ +1 for mentioning the conditional normality, not normality of the pooled distribution of the response variable. $\endgroup$
    – Dave
    Sep 2, 2020 at 3:35
-1
$\begingroup$

Definitely not wrong to use MSE. MSE penalizes large errors more than small errors. This means more penalization for predicting a target in the wrong modal.

If the network doesn't converge with advanced optimizers like adam or nesterov, you can always build a hierarchical model. First predict which mode it belongs to via logloss. Then have two models, one for each mode.

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.