1
$\begingroup$

In this question linked below, it was addressed why we would center the features in linear regression.

When conducting multiple regression, when should you center your predictor variables & when should you standardize them?

What I understood was that its because it gave the bias term meaning - it made it the predicted value of the label when all the features were at their means.

If $y$ is the value of the label, it gave the intercept with the $y$ axis meaning.

However, would there ever be any reason for centering the labels? What would that even mean, graphically and intuitively?

Thanks!

$\endgroup$
0
$\begingroup$

Say you have a dataset with 2 features, $X$, and $Y$.

Centering the features of this dataset would mean we make it so that the mean value of the new $X$ feature becomes $0$, and the mean value of the new $Y$ feature becomes $0$.

In other words, if we get a dataset where the $X$ and $Y$ feature-values are centered, that means that they're given in terms of the distance from the most average feature-values for that dataset. When $X$ is $0$ and $Y$ is $0$, that corresponds to the "most average" datapoint.

Now, let's say we want to predict $Y$ based on $X$ by doing linear regression on this data-set. In other words, we are turning $Y$ into the label we want to predict, and $X$ into the single feature we're predicting it with.

Now, what happens? If we don't center either of the random-variables, then our linear regression model will simply fit a line to the $Y_s$ based on the $X_s$. $$y=mx+b$$.

That's okay. However, in this case $+b$, the intercept, has absolutely no meaning. It's the label which our model predicts a datapoint would take when $X=0$. However, $X$ might correspond to a feature where equaling zero is absolutely senseless! For example, what if $X$ is the height of a newborn baby?

By centering $X$, the bias now has meaning. Since $X$ is centered, it now takes on a value of $0$ for its average value. That makes the bias, $+b$, equal to the $Y$ feature-value for the baby with "the most average height!!!" Pretty cool!

Now, what if we center the $Y$ labels as well? What happens to the bias - what does it represent?

First of all, notice what happens to the linear if we center the $Y$ labels. Centering the $Y$ labels means that we subtract the mean of $Y$ from each $y$ value, so that the value that was previously the mean.

Let $X'$ be the centered feature, $Y$ be the uncentered label, and $Y'$ be the centered label.

If all we're doing is setting $Y'=Y-\mu_y$, then our linear equation should go from...

$$y=mx'+b$$

...to...

$y'=mx'+(b-\mu_y)$

...that is, all that should happen to our linear equation is that it should get shifted down by $\mu_y$.

Say that $Y$ corresponds to the weight of a newborn baby.

Before centering $Y$ and turning it into $Y'$, when $X'=0$ the linear equation told us what we should predict the weight of a newborn baby of average height to be. That was $b$.

Now, when $X'=0$, we instead get $b-\mu_y$. That is, we get the expected difference between the weight of a newborn baby of average height and the average weight of these babies!

Now, for each value of $X'$, what we're getting is the expected difference between the weight of a baby whose height is $X'$ away from the average height and the weight of the baby with the average weight!

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.