Simple Regression: how to prove that adding an observation that exactly follows the regression line never decreases the magnitude of the correlation? Suppose we fit by least square a regression line to $n$ pairs of $(x_i,y_i)$ observations, with
$$\hat{y}_i = \hat{\beta}_0 + x_i \hat{\beta}_1$$
Now suppose we add a single observation $(x_{n+1}, y_{n+1})$ that fits the regression line perfectly,
$$ y_{n+1} - \hat{y}_{n+1} = 0 $$
where $\hat{y}_{n+1}$ is calculated from the regression coefficients obtained from the previous $n$ observations only.
How can I prove that adding this new observation will never decrease the magnitude of the correlation $r$ between $x$ and $y$?
 A: The magnitude of the correlation cannot decrease by adding a value that perfectly fits the regression line.
First, we note that the extra observation can't change the sign of the simple regression slope. This is because the slope $\hat{\beta}_1$ won't be affected by the new observation, and the sample correlation $r$ equals $\hat{\beta}_1$ times a positive constant. Without loss of generality, we assume $r \ge 0$.
Next, consider the coefficient of determination $R^2$. In simple linear regression, the coefficient of determination $R^2$ is the square of the correlation $r$ (see this proof for details).
Because the square is monotonous function over the $[0, 1]$ interval, proving that the magnitude of the correlation cannot decrease is equivalent to proving that $R^2$ cannot decrease.
Denote as $R^2_n$ the coefficient of determination, and $\bar{Y}_n$ the mean of $Y$, computed from the first $n$ data points. We can write
\begin{align} R^2_n &= 1 - \frac{\sum^{n}_{i=1} ({Y}_i - \hat{Y}_i)^2 }{\sum^{n}_{i=1} ({Y}_i - \bar{Y}_n)^2} \\ &= 1 - \frac{\sum^{n}_{i=1} ({Y}_i - \hat{Y}_i)^2 }{\sum^{n}_{i=1} (\hat{Y}_i - \bar{Y}_n)^2 + \sum^{n}_{i=1} ({Y}_i - \hat{Y}_i)^2} \end{align}
Now, consider the coefficient of determination $R^2_{n+1}$ and mean $\bar{Y}_{n+1}$ based on the $n+1$ observations. We can write
\begin{align} R^2_{n+1} &= 1 - \frac{\sum^{n+1}_{i=1} ({Y}_i - \hat{Y}_i)^2 }{\sum^{n+1}_{i=1} (\hat{Y}_i - \bar{Y}_{n+1})^2 + \sum^{n+1}_{i=1} ({Y}_i - \hat{Y}_i)^2} \\
&= 
1 - \frac{\sum^{n}_{i=1} ({Y}_i - \hat{Y}_i)^2 }{\sum^{n+1}_{i=1} (\hat{Y}_i - \bar{Y}_{n+1})^2 + \sum^{n}_{i=1} ({Y}_i - \hat{Y}_i)^2 } 
 \end{align}
To prove that the magnitude of the correlation doesn't decrease, it is sufficient to prove that
$$ \sum^{n+1}_{i=1} (\hat{Y}_i - \bar{Y}_{n+1})^2 \ge  \sum^{n}_{i=1} (\hat{Y}_i - \bar{Y}_n)^2$$
To do so, we see that that
$$ \sum^{n+1}_{i=1} (\hat{Y}_i - \bar{Y}_{n+1})^2 
\ge
   \sum^{n}_{i=1} (\hat{Y}_i - \bar{Y}_{n+1})^2 
\ge 
   \sum^{n}_{i=1} (\hat{Y}_i - \bar{Y}_{n})^2 
$$
Where the last inequality is true because $\bar{Y}_{n}$ is the minimizer of  $\sum^{n}_{i=1} (\hat{Y}_i - k)^2$.
This proves that the magnitude of the correlation can't decrease. We also note that the previous inequalities are strict when $x_{n+1} \ne \bar{x}_n$; we conclude that the correlation will not change if and only if either $x_{n+1} = \bar{x}_n$ or $r = 0$, and that the correlation will increase otherwise.
