My question is related to this question but I fail to transfer the answers to a multiple regression setup.

Suppose, a matrix $Z$ consists of $K$ columns of $T$-vectors together with a constant column. Further, let the vector $Y\in\mathbb{R}^T$ be such that regressing $Z$ on $Y$ yields zero slopes except for the intercept term:

So, computing OLS estimate $\beta^Y$ for $$Y = Z\boldsymbol{\beta^Y} + u_Y$$ yields $\beta^Y = [\hat{Y}, \boldsymbol{0}]$ where $\hat Y$ is the sample mean of $Y$.

Define now for any $i$ the vector $V_i := Y \circ Z_i$ (where $\circ$ is the element-wise multiplication of the two vectors), where $Z_i$ is the i-th column of the matrix Z.

My question: Does our knowledge about the relation between $Y$ and $Z$ help to give a formula for the OLS coefficient of the regression $$V_i = Z\boldsymbol{\beta} + u_V$$ in terms of sample moments of $Z$ and $Y$?

Univariate Case If we assume $K=2$ ($Z$ consists of one constant + one additional variable), then, (using the answer of @Jarle Tufto) making use of the law of total covariance and independence of $Z_2$ and $Y$ one gets: $$\beta_2 = \frac{\mbox{Cov}(Z_2,V)}{\mbox{Var}(Z_2)} = \frac{E(Y)\mbox{Var}(Z_2)}{\mbox{Var}(Z_2)} = E(Y)$$ Replacing with sample counterparts yields $\beta_2 = \hat{Y}$.

Small simulation example to show that the relation works:

K <- 1
T <- 150000
Z <- cbind(1,matrix(rnorm(K*T),nc=K))

# Artifical creation of Y
Y_raw <- rnorm(T)
beta_sample <- solve(t(Z)%*%Z)%*%t(Z)%*%Y_raw

Y <- 1+Y_raw - Z%*%beta_sample
beta <- solve(t(Z)%*%Z)%*%t(Z)%*%Y

# Create Vi
i <- 2
Vi <- Y*Z[,i]
betaV <- solve(t(Z)%*%Z)%*%t(Z)%*%Vi
             [,1]      [,2]
[1,] 2.671577e-06 0.9983683

Multivariate Case Here I am in trouble - how can I derive the regression coefficients for $k>2?

$$[\alpha, \beta]' = (X'X)^{-1}X'V_i$$

  • $\begingroup$ Your notation is obscure and ambiguous. What kind of object is "$Z_i$" and how is it related to $X,Y,Z,V$? $\endgroup$ – whuber Nov 20 '17 at 21:02
  • $\begingroup$ thanks @whuber and apologies for the strange notation, I hope the clarification helps. $\endgroup$ – muffin1974 Nov 20 '17 at 23:16
  • $\begingroup$ Thanks. Are you trying to say that $Z$ consists of $K$ columns of $T$-vectors, together with a constant column? (The notation doesn't say that, but if it's misread in certain ways it suggests this might be what you mean.) If so, then evidently $V$ is independent of all the $Z_j$ for $j\ne i$, so this doesn't seem to be a multivariate problem at all. $\endgroup$ – whuber Nov 20 '17 at 23:22
  • $\begingroup$ Thanks @whuber! I learn a lot by just updating my question, I am sorry for the misleading notation. I hope it is fixed now. I think in your comment you stated correctly what I was trying to say. However, why should $V_i$ be independent of all the $Z_j$ for $j \neq i$, as the $Z_i$s do not necessarily have to be independent? $\endgroup$ – muffin1974 Nov 21 '17 at 9:33
  • $\begingroup$ Your original phrasing asserted that $(X,Y)$ was independent: that implies the $X_i$ are independent of each other, too. Your new phrasing tries to relax that assumption. However, it's unclear, because now $Y$ must be a random variable but $Z$ is merely a bunch of "columns". In what sense, then, could $Y$ be independent of $Z$? Do you mean that $Z$ is some kind of random $T\times K$ matrix? These details matter because they determine how to interpret this question. $\endgroup$ – whuber Nov 21 '17 at 14:32

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Browse other questions tagged or ask your own question.