Why do these independant variables have significant explanatory power, when 'theoretically' they should have none? (Self contained example inside)

Question

I am putting together a model which involves a simple linear regression, and to aid the development I have put together a process for generating synthetic observations.

The idea is that you have several variables which evolve according to an unobservable continuous time multivariate process. Each variable can only be sampled once a day, but not all at the same time. Some variables are sampled at 8am, and some at 8pm. The key point is that the same continuous time process under the hood drives all variables. My task is to try to discover the nature of that hidden process.

As part of this, I need to fit a model of 8am (early) variables regressed on lagged observations of 8pm (late) variables, because there is some overlap from when the late variables from t-1 are evolving and the early variables from t are evolving. Remember: the same underlying process drives the variables, they are just sampled out of phase which introduces auto-correlation in the observations. The underlying process itself does not have any auto-correlation.

$$x_{t,early} = A.x_{t-1,late} + \epsilon_t$$

So far this is all fairly straight forward and it is easy to fit this model. I have the following R code to generate synthetic observations and fit the model.

library(magrittr)
library(MASS)
library(xts)
rm(list = ls())
cv =
  matrix(0.4, nrow = 6, ncol = 6) %T>%
  {diag(.) <- 1}

N = 50001
x = mvrnorm(n=N, mu = rep(0,6), Sigma = cv) %>% zoo

earlyIndex = ceiling((1:N)/2) %>%
  {(.) * 60 * 60 * 24} %>% as.POSIXct(origin = "2020-01-01", tz = "UTC") %>% as.Date
lateIndex = ceiling(((1:N) - 1)/2) %>%
  {(.) * 60 * 60 * 24} %>% as.POSIXct(origin = "2020-01-01", tz = "UTC") %>% as.Date

x_early = x[,1:3] %>% aggregate(earlyIndex, sum) %>% xts %T>% {colnames(.) <- paste0("early ", 1:3)}
x_late = x[,(1:3) + 3] %>% aggregate(lateIndex, sum) %>% xts %T>% {colnames(.) <- paste0("late ", 1:3)}

joined = merge(x_early, lag(merge(x_early, x_late, join = "inner"), na.pad = F), join = "inner")

lmAr1 = lm(joined[,1:3] ~ joined[,-(1:6)] - 1)
summary(lmAr1)

The idea is serially independent observations $x$ are drawn from a multivariate distribution. These observations represent the hidden process if the entire set can be sampled at both 8am and 8pm. I then aggregate the early $x_{early}$ and late $x_{late}$ out of phase. The results show a highly significant fit with all p values essentially 0.

But then, as a sense check, I wanted to check that if I include lagged early regressors in the model, these regressors would not have significant explanatory power. Intuitively, this seems to make sense because the underlying process generates serially independent observations. Indeed, in the model $x_{t,early} = B.x_{t-1,early} + \epsilon_t$ the values in $B$ are all insignificant.

However, if we define $x_{all}=x_{early}|x_{late}$ to be the inner join of all the observations, then we fit the following model

$x_{t,early}=C.x_{t-1,all} + \epsilon_t$

lmAr2 = lm(joined[,1:3] ~ joined[,-(1:3)] - 1)
summary(lmAr2)

Then rather confusingly, for me at least, we see all regressors, including lagged early observations, playing an extremely significant role. Here is the output for the first regressand.

Response early.1.1 :

Call:
lm(formula = early.1.1 ~ joined[, -(1:3)] - 1)

Residuals:
       early.1.1
Min    -4.950529
1Q     -0.929542
Median -0.006248
3Q      0.919383
Max     5.146397

Coefficients:
                         Estimate Std. Error t value Pr(>|t|)    
joined[, -(1:3)]early.1 -0.046012   0.006967  -6.604 4.07e-11 ***
joined[, -(1:3)]early.2 -0.049209   0.006955  -7.075 1.53e-12 ***
joined[, -(1:3)]early.3 -0.040320   0.006976  -5.780 7.56e-09 ***
joined[, -(1:3)]late.1   0.120096   0.006931  17.328  < 2e-16 ***
joined[, -(1:3)]late.2   0.121404   0.006940  17.493  < 2e-16 ***
joined[, -(1:3)]late.3   0.125071   0.006974  17.934  < 2e-16 ***
---
Signif. codes:  0 ‘***’ 0.001 ‘**’ 0.01 ‘*’ 0.05 ‘.’ 0.1 ‘ ’ 1

This doesn't make sense to me, but maybe I have forgotten something important about linear regression since I learned about it all those years ago. Why do the extra regressors have extremely significant coefficients?

UPDATE Further to baer's answer, the conditional expectation can be calculated as

sig = c(2,0,0.4,0,2,0.4,0.4,0.4,2) %>% matrix(nrow = 3)
sig_xy = sig[1, 2:3]
sig_yy = sig[2:3,2:3]

sig_xy %*% solve(sig_yy)

For a proof, see theorem A.3 in this pdf.

Answer 1

Write the three dimensional vector $v \sim \mathcal{N} \left( \begin{bmatrix} 0 \\ 0 \\ 0 \\ \end{bmatrix}, \begin{bmatrix} 2 & 0 & 0.4 \\ 0 & 2 & 0.4 \\ 0.4 & 0.4 & 2 \end{bmatrix} \right)$. Notice that this is the distribution of the rows of e.g. joined[,c(1,4,7)].

Here, we have that $v_1$ depends on $v_3$ and has correlation $0.2$: this just expresses that the current-time early observation ($v_1$) depends on the previous-time late observation ($v_3$). If we do the the regression and look at lm(joined[,1] ~ joined[,7] - 1), we'll rightly see a significant coefficient. Also, we have that $v_1$ does not depend on $v_2$: this expresses that the current-time early observation ($v_1$) does not depend on the previous-time early observation ($v_2$). If we do the the regression and look at lm(joined[,1] ~ joined[,4] - 1), we'll see an insignificant coefficient.

However, marginal independence is not the same as conditional independence. In particular, here we have that $$\mathbb{E} \left[ v_1 | v_2, v_3 \right] = \frac{-1}{3.84} (.16 v_2 - .8 v_3) \approx 0.2083 v_3 - 0.0417 v_2,$$ showing that $v_1$ is not conditionally independent of $v_2$ given $v_3$. Indeed, when we run lm(joined[,1] ~ joined[,c(4,7)] - 1), these are the coefficients we get (up to sampling error).

The same analysis can be done when including all columns (rather than just the 1,4,7) and similar conclusions will be found, but focusing on these columns allows the point to most clearly be illustrated.