NN works well on preproccessed data, but results are poor after de-normalizing - predicting non-normalized prediction values using NN?

Question

So I'm trying to build a neural network to fit an approximating function to a data set.

The data set consists of (input,output) paris where the input features are discrete numbers [0,1,2,...,1027].

While the output is a single continous number between [0, 1027).

Now, If in preprocessing I normalize both my input features and outputs by dividing by the max value in the data set, my neural network which consists of a single hidden layer (relu) and a regression layer in the output learns well and I'm getting my MSE to drop to a nice level and I can plot and see that the NN fits well both the train set and the test set.

The problem is, when I de-normalize back the estimations to their [0,1027) range, the erros get multiplied by the max value (1027) and the overall results are now very poor.

Any tips on dealing with this kind of problem? how can I preform pre proccesing so my NN learn well, but not hurting performance when de-normalizing.

Thanks.

Answer 1

Any tips on dealing with this kind of problem? how can I preform pre proccesing so my NN learn well, but not hurting performance when de-normalizing.

You have done a good job of preprocessing the data by scaling it with Min and Max. You can also try subtracting the mean of the column and dividing with the standard deviation of the column. After preprocessing, do the train-test split of the data.

I want to point out that the conclusion that you have made that the performance is being hurt when de-normalising is not correct in the sense that the you are comparing same things (MSE for Normalized predictions vis-à-vis MSE for de normalized prediction) with very different scales.

The issues you have here are two:

A) Fine tuning your neural network. You can tune the hyperparameter of the neural network to achieve superior performance.

B) Coming up with an appropriate measure of accuracy. I would suggest a relative error measure. You can use Relative Squared Error(RSE). This measures the goodness of a prediction as compared to predicting the mean of the variable all the time. This measure can be used to compare models across data sets.