Part 1
If $y=\alpha+x\beta+\epsilon$ and $E[\epsilon]=0$, then $E[y]=\alpha+E[x]\beta$, therefore $\alpha = E[y]-E[x]\beta$.
If you assume that $y$ and $x$ have the same marginal distribution, then $E[y]=E[x]=\mu$, and
$$\alpha=(1-\beta)\mu,\qquad y=(1-\beta)\mu+x\beta$$
i.e. $y$ is a weighted average of the population mean $\mu$ and the predictor.
But if $\text{var}(y)=\text{var}(x)$, then $\beta=\frac{\text{cov}(x,y)}{\text{var}(x)}=\text{corr}(x,y)$. This is where $(1-r)$ comes from.
Since $\beta<1$ (unless $\beta$=1, a degenerate case), $y$ is on average closer to the population average than $x$.
Galton observed that extreme heights in parents are not passed on completely to their offspring. Rather, the heights in the offspring regress towards the population average. Galton called this phenomenon regression towards mediocrity.
Let's use Galton's data (https://github.com/vincentarelbundock/Rdatasets/raw/master/csv/mosaicData/Galton.csv):
> h <- read.csv("Galton.csv")
> head(h)
X family father mother sex height nkids
1 1 1 78.5 67.0 M 73.2 4
2 2 1 78.5 67.0 F 69.2 4
3 3 1 78.5 67.0 F 69.0 4
4 4 1 78.5 67.0 F 69.0 4
5 5 2 75.5 66.5 M 73.5 4
6 6 2 75.5 66.5 M 72.5 4
Since men are taller than women on average, Galton replaced the two parents' heights with the average of the father's and mother's heights, the latter scaled up by $1.08$, and multiplyed daughters' heights by $1.08$ (see here):
> h$midparent <- (h$father+h$mother*1.08)/2
> h$midchild <- ifelse(h$sex=='M', h$height, h$height*1.08)
> mean(h$midparent)
[1] 69.22201
> mean(h$midchild)
[1] 69.23371
Notice that you can assume $E[y]=E[x]=69.2$. You can now use several models, but let's keep it simple:
> reg <- lm(midchild ~ midparent, data=h) # simple linear regression
> reg$coefficients
(Intercept) midparent
18.7669821 0.7290562
> alpha <- as.numeric(reg$coefficients[1])
> beta <- as.numeric(reg$coefficients[2])
If the average of the father's and mother's heights is above population average, then their children will be a bit shorter:
> x <- 75 # midparent height
> alpha + beta * x # average children height
[1] 73.4462
What if $x$ is lesser than $\overline{x}$? Their children will be a bit taller:
> x <- 65
> alpha + beta * x
[1] 66.15563
This is the regression to the mean.
Part 2
Let's come to pre-test and post-test. Follow David R. Shanks and generate the following data:
> s <- rnorm(100, 1, 1) # stimulus
> ex <- rnorm(100) # random error
> ey <- rnorm(100) # random error
> x <- 100 * s + 30 * ex # performance score
> y <- 0.30 * s + ey # awareness measure
> (r <- cor(x,y)) # y and x are less than perfectly correlated
[1] 0.352693
> summary(lm(y ~ x))$coefficients # y = a + bx + e
Estimate Std. Error t value Pr(>|t|)
(Intercept) 0.137141042 0.1436701078 0.9545552 0.3421513684
x 0.003686547 0.0009880176 3.7312569 0.0003194889
$s$ is a stimulus, $x=100s+30\epsilon_x$ is a performance score, some assessment of perception or memory, $y=0.30s+\epsilon_y$ is an
awareness measure, such as a verbal report or a forced-choice response (read Shanks's paper to better understand the experiment, I'm not a psychologist :)
Now convert $y$ and $x$ to standard normal variables $z(y)$ and $z(x)$, which have the same (marginal) distribution:
> zx <- scale(x) # mean = 0, sd = 1
> zy <- scale(y) # mean = 0, sd = 1
> (r <- as.vector(cor(x,y))) # unchanged
[1] 0.352693
> summary(lm(zy ~ zx))$coefficients
Estimate Std. Error t value Pr(>|t|)
(Intercept) 1.729815e-17 0.09405011 1.839248e-16 1.0000000000
zx 3.526930e-01 0.09452392 3.731257e+00 0.0003194889
Notice that $\hat\beta=r$.
When performance and awareness measures are collected, usually performance show some above-chance sensitity to $s$, but awareness does not. This is why researchers have turned to so-called post hoc selection: data from partecipants whose awareness measure is below some cutoff are analyzed separately. If their mean performance score is greater than an appropriate baseline for that test, then it is concluded that some true unconscious cognition has been demonstrated.
Actually, post hoc analysy does not show anything else than regression to the mean. Let $c$ be a cutoff on $zy$, e.g. $c=\overline{zy}=0$. If you select $zy$ values lesser than $c$, and the corresponding $zx$ values, you'll always get $zx$ values closer to the mean than $zy$ values:
> cutoff <- mean(zy) # cutoff = 0, you can change it
> y_rtm <- zy[which(zy <= cutoff)] # awareness measure <= cutoff
> x_rtm <- zx[which(zy <= cutoff)] # corresponding performance score
> mean(y_rtm) # low awareness measure
[1] -0.7361932
> mean(x_rtm) # performance score closer to the mean
[1] -0.266041
The same result if you select $zy$ values greater than $c$:
> y_rtm <- zy[which(zy >= cutoff)] # awareness measure <= cutoff
> x_rtm <- zx[which(zy >= cutoff)] # corresponding performance score
> mean(y_rtm) # high awareness measure
[1] 0.7361932
> mean(x_rtm) # performance score closer to the mean
[1] 0.266041
Another example of regression to the mean. It's just like selecting shorter parents (parents whose height is below some cutoff) and observing taller children, or selecting taller parents and observing shorter children.
Clearly, the same happens if you use nonscaled variables: "When two variables are imperfectly correlated [...] it is a statistical certainty that applying an extreme cutoff on one dimension [...] will yield a less extreme cutoff for the expected value of the other variable" (Shanks). Scaled variable help understand that the extent of regression to the mean depends on the correlation $r$, because when scaling (generally speaking, when $y$ and $x$ have the same distribution) $\hat\beta=r$.
Addendum
So, would you please describe (possibly in words) how the last part of
my description above implies that mean of $y$ (post-test) will shift to
the mean of $x$ (pre-test, but here their common mean as $E[y]=E[x]$) by
$(1−r)$?
Look at Galton's data. You have $E[y]=E[x]=69.2$, and:
a) if $x=75$, then $\hat y=\hat\alpha+\hat\beta 75\approx 73.4\approx 75+(1-r)(69.2-75)$;
b) if $x=65$, then $\hat y=\hat\alpha+\hat\beta 65\approx 66.1\approx 65+(1-r)(69.2-65)$.
where $r=\hat\beta$. The distance between $x$ and $\mu$ is $|\mu-x|$, $\hat y$ gets closer to $\mu$ by $(1-r)|\mu-x|$. Why? Because:
\begin{align*}
\hat\alpha+x\hat\beta &= (1-\hat\beta)\mu+x\hat\beta \\
&=\mu-\mu\hat\beta+x\hat\beta+x-x =x+(1-r)\mu-(1-r)x\\
&=x+(1-r)(\mu-x)
\end{align*}
References
Bruce E. Hansen, Econometrics, §2.26
Stephen M. Stigler, The History of Statistics: The Measurement of Uncertainty before 1900, Chap. 8
James A. Hanley, "Transmuting" Women into Men: Galton’s Family Data on Human Stature, The American Statistician, 2004, 58(3). Hanley compares sharper models.
David R. Shanks, Regressive research: The pitfalls of post hoc data selectionin the study of unconscious mental processes, Psychonomic Bulletin & Review, 2017, 24.
