It's multivariate linear regression meaning multiple dependent variables (Y). Data A has both X (explanatory variables) and Y, but data B has only Y.

I wonder if there is any way to incorporate data A and B into the regression model.

If I remember correctly, Statistical Analysis with Missing Data by Little et al, suggested some kind of iterative approach like EM algorithm where you repeat following until convergence

  1. Estimate missing X based on current regression model
  2. Compute regression model with full X estimated in #1

But in this case, I have all columns of X matrix is missing. So my questions:

  1. Is the approach above still effective? That is, would including data B into the model help in any way?
  2. If so, how missing full columns of X can be estimated?

Update: The reason I want to add data B is because data B is application-specific data while data A is more general data. I hope to have training data that has more "weights" for application-specific input space.

  • 1
    $\begingroup$ If I properly understand, a full column of X missing means that you did not observe an explanatory variable at all. I do not believe it is possible to estimate it unless there is a known relationship with the observed covariates. This actually happens all the time because there are covariates that you do not observe (for a number of possible reasons). $\endgroup$
    – user10525
    Apr 20, 2012 at 13:15
  • $\begingroup$ @Procrastinator I think we can do regression X on Y for the missing data estimation. $\endgroup$ Apr 20, 2012 at 13:23
  • $\begingroup$ Are you interested on a calibration model? Otherwise it makes no sense to estimate $X$ and then use it on a regression model for $Y$. $\endgroup$
    – user10525
    Apr 20, 2012 at 13:29
  • $\begingroup$ @Procrastinator As I said on my update above, I hope to have training data that has more "weights" for application-specific input space. $\endgroup$ Apr 20, 2012 at 13:33

1 Answer 1


In this case, dataset B only provides information about the unconditional distribution of the responses (Y) and nothing about the relationship between the atributes (X) and the responses (Y). So in this case I rather doubt there will be much to gain from using dataset B to estimate the regression parameters, which encode this relationship, without making some stronger assuption about the data generating process.

One condition where it might be of some use is in the presence of covariate shift, where there has been a change in the distribution of the attributes between the time the data were collected and in operational conditions. In this case if B is representative of operational conditions it may be of some use.


Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge that you have read and understand our privacy policy and code of conduct.

Not the answer you're looking for? Browse other questions tagged or ask your own question.