# What is the relationship between linear regression and z score regression?

So I'm taking a stat's class that has introduced z-score regression. According to my professor, z-score regression gives us the "line of best fit" when the data has a linear structure.

I've previously taken a machine learning course where I learned that the "line of best fit" was obtained through linear regression.

My question then is are z-score regression and linear regression the same thing?

Here's where my mind is at: Let $$r$$ be our correlation between our variables $$x$$, the data we have, and $$y$$, what we're trying to predict.

Correlation is $$r = {\frac{1}{n - 1}} \sum_{i = 1}^{n} Z(x_i)Z(y_i)$$ where Z() is the z score function.

Then trying to predict a $$\hat{y}$$ from using our z score regression line at $$x = \hat{x}$$

$$Z(\hat{y}) = r Z(\hat{x})$$

$$\frac{\hat{y} - \bar{y}}{SD(y)} = r \frac{\hat{x} - \bar{x}}{SD(x)}$$ Where $$\bar{y}$$ and $$\bar{x}$$ are the respective averages.

So isolating $$\hat{y} = r \frac{SD(y)}{SD(x)} \hat{x} + \bar{y} - r \frac{SD(y)}{SD(x)} \bar{x}$$

I'm not sure what insights I'm supposed to get from this...(I'm embarrassed lol)...

I guess if we attempt to measure how well this z-score regression line does in terms of how close it is to our data:

$$min \sum_{i = 1}^{n} (y_i - (a_0 + a_1 x_i))^2$$This is how far the actual data is to our z-score line

Let's attempt to find $$a_0$$ and $$a_1$$ which after the algebra we'll get:

$$a_1 = r \frac{SD(y)}{SD(x)}$$ $$a_0 = \bar{y} - b_1\bar{x}$$

which will give us exactly the formula for $$\hat{y}$$

In my machine learning class we learned:

$$\hat{y} = f(x;\theta, \theta_0) = \theta^T x + \theta_0$$

If we take our cost function/objective function to be $$\sum_{i = 1}^{n} (y_i - (\theta_0 + \theta^T x_i))^2$$...

Then looking for the best $$\theta$$ and $$\theta_0$$, we should get the same thing as we did for $$a_0$$ and $$a_1$$. So if we use the same cost function/objective as we do in Z-Score regression... then the two approaches are exactly the same?

Is there something special about this? Is this question just non-sensical?Am I supposed to be able to gain some special insight between these two subjects(stats and ML)?

• "z-score regression" is nonstandard terminology, we call it linear regression in stats too (and have for decades before the name "machine learning" appeared :)). Z-scores are these strange subjects that appear only in intro classes. Aug 21, 2022 at 16:43