Why this OLS fitting will converge to (0,-0.5)?

Question

In this AR1 model, if we fit y with diff(y), regardless the true coefficient, when N is large, it seems the model will converge to (0,-0.5), why?

> set.seed(0)
> ts.sim <- arima.sim(list(order = c(1,0,0), ar = c(0.1)), n = 1e6)
> x= diff(ts.sim)
> y=ts.sim[-1e6]
> lm(y~x)

Call:
lm(formula = y ~ x)

Coefficients:
(Intercept)            x  
 -4.376e-05   -5.000e-01  

> ts.sim <- arima.sim(list(order = c(1,0,0), ar = c(0.9)), n = 1e6)
> x= diff(ts.sim)
> y=ts.sim[-1e6]
> lm(y~x)

Call:
lm(formula = y ~ x)

Coefficients:
(Intercept)            x  
  -0.003551    -0.500002  

Answer 1

This is a law of large numbers in action.

Let $\rho$ be the parameter (the lag-1 correlation) and let $\varepsilon_i$ be a sequence of iid standard Normal variables, so that for $i=1, 2, \ldots,$ the model is

$$y_{i+1} = \rho y_i + \varepsilon_{i+1}$$

and $y_0=0.$ Therefore the first differences are

$$x_{i+1} = y_{i+1} - y_i = (\rho-1)y_i + \varepsilon_{i+1}.$$

Because $y_0=0,$ inductively the $x_i$ and $y_i$ all have expectations of zero.

Because $\rho \lt 1,$ notice that $x$ and $y$ move in opposite directions: they must be negatively correlated. You might (intuitively) think that the amount of correlation depends on $\rho.$ However, it also depends on the variances of the $x_i$ and $y_i.$ We need to examine these.

As in the question, stop the series at $n$ and regress $y$ against $x.$ The slope estimate is

$$\hat \beta = \frac{\sum_i (y_i-\bar y)(x_i - \bar x)}{\sum_i (x_i-\bar x)} \approx \frac{\sum_i y_i x_i}{\sum_i x_i^2}$$

because, for large $n,$ the means $\bar y$ and $\bar x$ will be close to their expectations, which are zero.

Again because $n$ is large, the empirical values in the fraction can be approximated by their expectations (a law of large numbers),

$$\frac{\sum_i y_i x_i}{\sum_i x_i^2} \approx \frac{E\left[\sum_i y_i x_i\right]}{E\left[\sum_i x_i^2\right]}.$$

Find those expectations from the definition of the series $(y_i).$ I will begin with a useful auxiliary calculation:

$$\begin{aligned} E\left[\sum_i y_i^2\right] &= \sum_i E\left[y_i^2\right] = \sum_i E\left[(\varepsilon_{i+1} + \rho y_i)^2\right]\\ &= \sum_i E\left[(\varepsilon_{i+1})^2\right] + 2\rho \sum_i E\left[\varepsilon_{i+1} y_i\right] + \rho^2 \sum_i E\left[ y_i^2\right]\\ &= n + 0 + \rho^2 E\left[\sum_i y_i^2\right] \end{aligned}$$

which is justified because the $\varepsilon_i$ have zero expectation, unit variance, and are independent of each other (whence, in particular, $\varepsilon_{i+1}$ is independent of $y_i$). Solving this equation gives

$$E\left[\sum_i y_i^2\right] = \frac{n}{1-\rho^2}.$$

With this (standard) result in hand we obtain, using similar calculations,

$$E\left[\sum_i y_i x_i\right] = -\frac{n}{1+\rho}$$

and

$$E\left[\sum_i x_i^2\right] = \frac{2n}{1+\rho},$$

whence

$$\hat \beta \approx \left(-\frac{n}{1+\rho}\right)\,/\,\left(\frac{2n}{1+\rho}\right) = -\frac{1}{2}.$$

(You can carry out this analysis rigorously using the Delta method, which applies because the denominator of $\hat\beta,$ $\sum_i x_i^2,$ will stay away from zero.)

Finally, because the linear regression passes through the point of averages and the means of the $x_i$ and $y_i$ are each close to $0,$ the constant term will be close to $0,$ too.