Suppose that $X_i, i=1,\ldots,n$ are some random variables. I'd like to do multiple linear regression to learn to predict any of these variables from the others. My model for the reconstructed variables is the following. $$X_i = \sum_{j=1}^n w_{i,j} X_i + \epsilon_i, \forall i=1,\ldots,n$$ A simple thing to do would be to tune the weights, $w$, to minimize the squared error between the reconstruction and the original. Obviously, there is a perfect but trivial solution! We have to enforce that $w_{i,i}=0$ so that we can't predict $X_i$ from itself. After imposing this constraint, we have what I'd call a self-regression problem. Has this simple model been studied and can anyone point me in the right direction?
A related problem would be to use dimensionality reduction to carry out this reconstruction. E.g., we could use an autoencoder with linear encoder and decoder, or simply use PCA. One problem with this point of view is that it is not completely straightforward to predict variable $X_i$ from the other variables $X_{j\neq i}$. Why not? Because the PCA vectors depend on $X_i$ in the first place. Again, I believe this context must be well-studied. We are just doing linear dimensionality reduction with missing variables and we want to know how well we can reconstruct the missing variables. Has this situation been studied?