What is difference between correlation and simple linear regression (binary dependent variable and continuous independent variable)? Many similar questions here, but this question combines two threads: relationship between correlation and regression, and correlation between a binary and a continuous variable.

*

*Since the relationship between a binary variable and a continuous variable is not linear, is Pearson correlation ever an appropriate measure of whether these two variables move together? Several sources mention that Point Biserial is equivalent to Pearson correlation in case one variable is binary and the other continuous. So is point Biserial correlation identical to Pearson correlation in this case? Is Point Biserial more appropriate/the most appropriate?

*Is it still the case that R-squared from OLS equals the square of Pearson correlation, if the dependent variable is binary and the independent variable is continuous? Why?

*Is there some type of regression between these variables for which R-squared is the square of Point Biserial correlation?

 A: As I remarked in a comment, the interesting question is (3): how might $R^2$ and the point biserial correlation (p.b.c.) coefficient be related?   In fact, all three questions will be answered in the following analysis.
We need formulas for the terms that go into computing $R^2$ and the p.b.c. coefficient, but since this focuses on a pair of variables $(X,Y)$ in which (say) $Y$ is binary, these formulas ought to be simple.  As usual, code $Y$ with values $0$ and $1.$  (The particular values don't matter, because correlations do not change when values are shifted or rescaled, but this set of values is conventional and easy to work with.)
Following Wikipedia, adopt the following notation:

*

*$n_0$ is the number of observations for which $Y=0$ and $n_1$ is the count of observations for which $Y=1.$


*$n = n_0 + n_1$ is the total number of observations.


*$\bar X = (X_1 + X_2 + \cdots + X_n)/n$ is the mean of the $X$ data.


*$\bar Y  = (n_0\times 0 + n_1\times 1)/n = n_1/n$ is the mean of the $Y$ data.


*Similarly, $M_0$ is the mean of the $X$ data corresponding to values where $Y=0$ and $M_1$ is the mean of the $X$ data for $Y=1.$

*

*Notice that $n_0 M_0 + n_1 M_1 = n \bar X$ because both expressions equal the sum of the $X$ data.


*The mean of an empty list of values is undefined.




*$n s^2 = (X_1-\bar X)^2 + (X_2-\bar X)^2 + \cdots + (X_n-\bar X)^n$ makes $s^2$ the (empirical) variance of the $X$ data (not the unbiased estimator!).

*

*I will write $s^2 = \operatorname{Var}(X)$ and $s = \operatorname{SD}(X)$ as a shorthand.


*The residuals are the values relative to their means: $X_i-\bar X$ for the $X$ data and $Y_i - \bar Y$ for the $Y$ data.




*The empirical variance of the $Y$ data is $$\operatorname{Var}(Y) = \frac{1}{n}\left[n_0(0 - \bar Y)^2 + n_1(1 - \bar Y)^2\right] =  \frac{n_0n_1}{n^2}.$$


*The empirical covariance of the $(X,Y)$ data is the mean product of the paired residuals.  In $n_0$ of the cases the $Y$ residual is $0-\bar Y=-n_1/n$ while in the remaining $n_1$ of the cases the $Y$ residual is $1-\bar Y = 1 - n_1/n = n_0/n.$  Consequently, $$\begin{aligned}\operatorname{Cov}(X,Y) &= \frac{1}{n}\left[ n_0 \frac{-n_1}{n}(M_0 - \bar X) + n_1 \frac{n_0}{n}(M_1 - \bar X)\right] = \frac{n_0n_1}{n^2}\left[M_1 - M_0\right].\end{aligned}$$
The point biserial correlation coefficient is defined to be
$$r_{pb} = \frac{M_1 - M_0}{s}\sqrt{\frac{n_0n_1}{n^2}} = \frac{\frac{n_0n_1}{n^2}\left[M_1-M_0\right]}{s\,\sqrt{n_0n_1/n^2}} = \frac{\operatorname{Cov}(X,Y)}{\operatorname{SD}(X)\operatorname{SD}(Y)} = \rho(X,Y)$$
where $\rho$ is the Pearson correlation coefficient.
Because $R^2(X,Y) = \rho(X,Y)^2$ (see the formulas at the end of https://stats.stackexchange.com/a/579154/919, for instance),

$$r_{pb} = \rho\quad \text{ and }\quad R^2 = r_{pb}^2 = \rho^2.$$

Incidentally, as you may check, whenever all the data are in one group ($Y=0$ only or $Y=1$ only) all three quantities--$R^2,$ $r_{pb},$ and $\rho$--are undefined.
