This is a specific application of the usual least squares derivation for linear regression but I'm confusing myself.

Suppose $X = (X_1, ..., X_d)$ is a random vector.

Pick any $i, j < d$ and let $X_{-ij}$ be $X$ without the $i$ and $j$ components.

Then we want to regress on $X_{-ij}$ to find $X_i$ and $X_j$.

So I want to find $\beta_i$ and $\beta_i^0$ such that $X_{-ij}^T\beta_i + \beta_i^0$ minimizes the expected least square error.

i.e, I want to find $\text{argmin}_{\beta_i, \beta_i^0}\mathbb{E}[(X_i - X^T_{-ij}\beta_i - \beta_i^0)^2]$ and similar for $\beta_j, \beta_j^0$

$\beta_i, \beta_i^0$ will be in terms of variances and covariances of the $X_k$ once I'm done.

I've been unable to get the terms to behave however, and have unable to find a solid expression for the $\beta_i$. Would appreciate help with the derivation.


1 Answer 1


The $ \beta_{i} $ elements are the correlation coefficients between $ {X}_{i} $ to all other elements.


Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge that you have read and understand our privacy policy and code of conduct.

Not the answer you're looking for? Browse other questions tagged or ask your own question.