# From LEAST SQUARE METHOD to Pearson cofficient and determination coefficient

the following procedure has the scope to describe how, from the Least square method, I obtained the Pearson coefficient and the determination coefficient. At the end of the procedure I have several questions.

If I apply the least square method, finding the residual $$r_{xi}$$:

$$r_{xi}=y_{i}-b_{x}x_{i}$$ (1)

The least square method consists, firstly, in doing the sum of the square of the residual:

$$\sum_{i}r_{i}^2=\sum_{i}(y_{i}-b_{x}x_{i})^2$$ (2)

If I want to find the minimum of the square of the residual, so I do the partial derivation and is equal to 0:

$$\frac{d\sum_{i}r_{i}^2}{db_{x}}=0$$ (3)

$$\frac{d\sum_{i}(y_{i}-b_{x}x_{i})^2}{db_{x}}=0$$ (4)

$$\sum_{i}(2b_{x} x_{i}^2-2 y_{i} x_{i})=0$$ (5)

$$b_{x} = \frac{\sum_{i} x_{i} y_{i}}{\sum_{i} x_{i}^2}$$ (6)

Viceversa, if I start from from this residual $$r_{yi}$$:

$$r_{yi}=x_{i}-b_{y}y_{i}$$ (7)

Doing the same procedure I obtain:

$$b_{y}= \frac{\sum_{i} y_{i}x_{i}}{\sum_{i} y_{i}^2}$$ (8)

If I do the product of $$b_{y}$$ and $$b_{x}$$ I obtain

$$b_{y} b_{x} = \frac{(\sum_{i} y_{i}x_{i})^{2}}{\sum_{i} y_{i}^2\sum_{i} x_{i}^2}$$ (9)

Finally if I do the square of this product I obtain:

$$\sqrt{b_{y} b_{x}} = \frac{\sum_{i} (y_{i}x_{i})}{\sqrt{\sum_{i} y_{i}^2\sum_{i} x_{i}^2}}$$ (10)

In doing the least square method, that I posted above , I noticed that if I substitute in (10) $$y_{i}$$ and $$x_{i}$$ with $$y’_{i}$$ and $$x’_{i}$$ which are equal to :

$$y’_{i}=y_{i}-\overline{y}$$ (11)

$$x’_{i}=x_{i}-\overline{x}$$ (12)

where $$\overline{y}$$ and $$\overline{x}$$ are the mean of the samples $$y_{i}$$ and $$x_{i}$$; $$\sqrt{b_{y} b_{x}}$$ is like the Pearson coefficient, whereas $$b_{y} b_{x}$$ is like the determination coefficient. So my questions are:

1. Is this the typical procedure to obtain the Pearson coefficient(PC) and the determination coefficent (DC)?
2. Why the PC and the DC take in consideration only the fluctuation of $$x$$ and $$y$$?
3. Theoretically, why taking the square root of the product of the angular coefficient ($$\sqrt{b_{y} b_{x}}$$ ) should have the meaning of the PC, i.e. a parameter that assesses the correlation between the two parameter taken in consideration? Thank you in advance
• numerator of $b_yb_x$ is incorrect in (9) Dec 5, 2020 at 12:41
• corrected! Thanks Dec 5, 2020 at 12:45
• You should square the entire sum Dec 5, 2020 at 13:11
• Thanks, corrected again :), Lukily the mistake does not affect the purpose of the answer. Do you have an idea of how to answer? Dec 5, 2020 at 13:15

The question is interesting. A possible explanation can be the fact that substituting in (10) $$y_{i}$$ and $$x_{i}$$ with $$y’_{i}$$ and $$x’_{i}$$ which are equal to :

$$y’_{i}=y_{i}-\overline{y}$$

$$x’_{i}=x_{i}-\overline{x}$$

Implicitly you are using a formula

$$y=bx+a$$

So the correlation coefficient take in account the possibility of an intercept a that has the formula:

$$a=\overline{y}-b\overline{x}$$

An other hint can be the fact that the formula (10) is a geometric mean.