# Standardisation and its effect on regression and the MSE

I am working on a linear regression problem using sklearn's diabetes dataset and coding things from scratch and while evaluating the performance of the model, I have come to question how to move between what the MSE is on the standardised dataset and what it would be on the unperturbed dataset.

I scaled both the X and Y variables and the algebra seems quite heavy and not sure working through it would be fruitful, hence I wanted to ask the community their thoughts.

The data are scaled according to the following formula, to give $$x_{s,i}$$ and $$y_{s,i}$$

\begin{align} x_{s,i} & = \frac{x_i - \bar{x}}{\sigma_{x}} \\[10pt] y_{s,i} & = \frac{y_i - \bar{y}}{\sigma_{y}} \end{align}

Then $$\hat{\alpha}$$ and $$\hat{\beta}$$, the MLE for the parameter's is given by \begin{align} \\ \hat{\alpha} & = \bar{y} - \hat{\beta} \bar{x} \\[10pt] \hat{\beta} & = \frac{s_{xy}}{s_{xx}} \\[10pt] & = \frac{\sum_{i=1}^N (x_i - \bar{x})(y_i - \bar{y})}{\sum_{i=1}^N (x_i - \bar{x})^2} \\[10pt] \end{align}

With the value of the fitted value given by \begin{align} \hat{y} & = \hat{\alpha} + \hat{\beta}x_i \\[10pt] & = \bar{y} - \hat{\beta}\bar{x} + \hat{\beta}x_i \\[10pt] \hat{y_{i}} & = \bar{y} - \hat{\beta} (x_i - \bar{x}) \end{align}

If we substitute the standardised variables into this formula then it becomes

\begin{align} \hat{\beta_s} & = \frac{\sum_{i=1}^N (x_{s,i} - \bar{x_s})(y_{s,i} - \bar{y_s})}{\sum_{i=1}^n (x_{s,i} - \bar{x_s})^2} \\[10pt] \end{align}

At this point I start looking at the algebra and think I may not find a quick and clean solution to this but I still want to understand how I can go between the MSE for unscaled data, $$MSE$$, and then also the scaled MSE, $$MSE_s$$. Plugging along I get down to the following and wanted to confirm if this is worth pursuing further

\begin{align} MSE_s & = \frac{\sum_{i=1}^N \big(y_{s,i} - \hat{y_{s,i}}\big)^2}{N} \\[10pt] & = \frac{\sum_{i=1}^N \big(y_{s,i} - \bar{y_s} + \hat{\beta_s} (x_{s,i} - \bar{x_s})\big)^2}{N} \\[10pt] & = \frac{\sum_{i=1}^N \big(y_{s,i} - \bar{y_s} + \frac{\sum_{i=1}^n (x_{s,i} - \bar{x_s})(y_{s,i} - \bar{y_s})}{\sum_{i=1}^n (x_{s,i} - \bar{x_s})^2}(x_{s,i} - \bar{x_s})\big)^2}{N} \\[10pt] & = \frac{\sum_{i=1}^N \big(y_{s,i} - \bar{y_s} + \sum_{i=1}^n (y_{s,i} - \bar{y_s})\big)^2}{N} \\[10pt] \end{align}

Would be grateful for anyone's thoughts / guidance.

I think you're working too hard. Standardizing the predictor ($$x$$) variables should have no effect on the overall model fit (provided it doesn't alleviate problems with numerical instability etc.); that is, it will change the parameter estimates but not the MSE, predictions, $$R^2$$, AIC, etc.. Centering the response variable (or otherwise shifting it) should have no overall effect (except on the predictions, which will be shifted by an identical amount), and scaling the response variable by a factor $$C$$ should scale the MSE by a factor $$C^2$$.
If you scale $$y$$ by $$1/s_y$$ (the sample standard deviation), then the MSE should end up being scaled by $$1/s_y^2$$.