Undo Batch Normalization in NN

I am using 2 BatchNormalization layers in Keras for a huge dataset that does not fit into memory. I can train with normalized values, but since I want to do a regression, I need to reverse or undo the effects of the normalization, that is, after I do

prediction = model.predict(my_test)


I obtain values that are affected by the 2 BatchNormalizations, and I want to apply to prediction a function to show the values in their natural scale. How can I do this in Keras? Note that I have 2 layers of BatchNormalization.

(I know how to do it when the dataset fits into memory and I normalize it before the training without using BatchNormalization, and then de-normalize it, but in this case I am confused). Thx.

Addendum: By @Romain's suggestion: full model. Inspired by https://medium.com/@rajin250/precipitation-prediction-using-convlstm-deep-neural-network-b9e9b617b436 I want to predict rainfall based in previous rainfall images for a 150x150 points of a grid area, so I have data from 0 to unbounded values,

model = Sequential()
return_sequences=True,
activation="tanh", recurrent_activation='hard_sigmoid',
kernel_initializer='glorot_uniform',
unit_forget_bias=True,
dropout=0.2, recurrent_dropout=0.2))

return_sequences=False,
activation="tanh", recurrent_activation='hard_sigmoid',
kernel_initializer='glorot_uniform',
unit_forget_bias=True,
dropout=0.2, recurrent_dropout=0.2))

#activation="sigmoid",



I have made explicit that the activation is commented since the sigmoid forces my values to be between 0 and 1. The training is working but the values I obtain are very small, far from reality, so my thought was that I needed to de-normalize somehow...

• I think you are having a mis conception about "Batch Normalization". This does not concern your input data but rather the output of each layer on a given batch of data. You don't need to "reverse" it. Can you give more detail about your architecture as well as your loss function please ? – Romain Mar 30 at 13:38
• I have updated the entry with the model and explanations. – David Mar 30 at 13:53
• If predicted values are not close to the observed values, this implies that the training loss is large. In other words, the model isn’t fitting the data very well. – Sycorax Mar 30 at 13:55
• But the model is not far from predicting well. Using a sigmoid activation in the last layer, I can see the rainfall in more or less correct places, although I don't have exact values since the sigmoid is limiting them, but it encourages me to think that the model is seeing something. Curiously, without a sigmoid, (as it should be with "mse" as loss and a regression) the training worsens. – David Mar 30 at 14:00
• I just saw the edit to your question. You should put a linear layer as the last part of the network. – Sycorax Mar 30 at 14:06