Denormalize prediction values for unseen data

Question

For my model, I need to apply z-normalization to the input data. For train and test, I can denormalize the output since I have the mean and standard deviation for both y-true train and test sets.

However, when using the same model for unseen data, I need to apply the same z-normalization step to the input data (each feature is z-normalized). My question is how to do the output's denormalization since the y-true set does not exist, and consequently I cannot have the mean and standard deviations.

Is saving the mean and standard deviation during the training step a valid approach to work around this issue? I know that an uncertainty level will be introduced, but I can't think about anything else.

I have seen other questions about denormalization, like this one here, but they are talking about a different problem.

Answer 1

It might be helpful to note if you have some training data $(X,Y)$ and a test point $x$ you wish to make a prediction for, you can take the following two approaches and you will get identical (not just in expectation but in value) results:

1: Perform a linear regression, regress $Y$ onto $X$. Your resulting parameter will be given by $X^{+}\cdot y$ and thus your prediction will be $X^{+}\cdot Y \cdot x^{T}$. Finally, subtract any constant $a$ from your prediction and divide by any other constant $b$ [note I'm using $x^{+}$ to denote the Moore-Penrose pseudoinverse]

2: Apply the transformation $\tilde{Y}=\frac{Y-a}{b}$. Regression $\tilde{Y}$ onto X. Then make your prediction, the result is $X^{+}\cdot \tilde{Y} \cdot x^{T}$

These two answers are identical (it's not entirely obvious, you need to use some properties of the pseudo-inverse to prove it)

The converse is also true that you could take your normalised target, do your regression, make a prediction and denormalise said prediction...or you could just denormalise the target (circular I know) and then do your regression and make the prediction, same result.

But the key is that the transformations you use have the be the same ones. The process of normalisation usually involves subtracting off the train mean and dividing by the train standard deviation, but more generally what I've said holds for any transformation of the form $y' = \frac{y-a}{b}$ and thus for any linear transformation.

In conclusion, de-normalise your prediction using the train mean and standard deviation.