1
$\begingroup$

I have build a feed forward neural network with 3 hidden layers for regression problem. The metrics I'm using for validation is MAPE. Following are the model parameters

#Define the model
NN_model = Sequential()

# The Input Layer :
NN_model.add(Dense(128, kernel_initializer='normal',input_dim = X_train.shape[1], activation='relu'))

# The Hidden Layers :
NN_model.add(Dense(256, kernel_initializer='normal',activation='relu'))
NN_model.add(Dense(256, kernel_initializer='normal',activation='relu'))
NN_model.add(Dense(256, kernel_initializer='normal',activation='relu'))

# The Output Layer :
NN_model.add(Dense(1, kernel_initializer='normal',activation='linear'))

# Compile the network :
NN_model.compile(loss='mean_absolute_percentage_error', optimizer='adam', metrics=['mean_absolute_percentage_error'])
##NN_model.compile(loss='mean_absolute_error', optimizer='adam', metrics=['mean_absolute_error'])
NN_model.summary()

The sample output looks like the following

Train on 18000 samples, validate on 4500 samples
Epoch 1/500
18000/18000 [==============================] - 3s 148us/step - loss: 672.5252 - mean_absolute_percentage_error: 672.5252 - val_loss: 29.3799 - val_mean_absolute_percentage_error: 29.3799

Epoch 00001: val_loss improved from inf to 29.37992, saving model to Weights-001--29.37992.hdf5
Epoch 2/500
18000/18000 [==============================] - 2s 133us/step - loss: 739.9019 - mean_absolute_percentage_error: 739.9019 - val_loss: 220.4918 - val_mean_absolute_percentage_error: 220.4918

Epoch 00002: val_loss did not improve from 29.37992
Epoch 3/500
18000/18000 [==============================] - 2s 129us/step - loss: 840.8005 - mean_absolute_percentage_error: 840.8005 - val_loss: 18.5716 - val_mean_absolute_percentage_error: 18.5716

My question is, in each epoch I see mean absolute percentage error and validation mean absolute percentage error. The later seems to be lower which is what I expect, Why is mean absolute percentage error so different and much higher than validation mean absolute percentage error?

Also, why is validation mean absolute percentage error fluctuating so much?

Appreciate any input.

$\endgroup$
  • $\begingroup$ I don't get why you expect the later (validation) to be lower. Is there an ordering in your data btw? $\endgroup$ – gunes Jan 30 at 9:56
  • $\begingroup$ Did you standardize your data ? this might be helpful stats.stackexchange.com/questions/255105/… $\endgroup$ – Harshit Mehta Feb 7 at 18:26

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Browse other questions tagged or ask your own question.