5
$\begingroup$

Very simple. I am looking for a package that does Multivariate Linear Regression with weights on the observations. Does anyone know of a package that does this? I am shocked that I have not been able to find any.

NOTE: R does NOT do multivariate regression. The lm() help page specifically states: "If response is a matrix a linear model is fitted separately by least-squares to each column of the matrix. " This means independent regression models for each response variable. Thus lm() does NOT do multivariate linear regression. It merely does several univariate linear regressions for convenience.

$\endgroup$
1
  • 1
    $\begingroup$ Although it is correct that lm() does not handle weighted multivariate regression, it does do unweighted multivariate regression properly. Fitting a least-squares estimate separately to each column of the response matrix provides the correct coefficient estimates. The "mlm" objects returned by lm() for models with response matrices contain the information needed for true multivariate inference. See Fox and Weinberg, and my further comments on an answer below. $\endgroup$
    – EdM
    Aug 11, 2020 at 12:19

3 Answers 3

2
$\begingroup$

Try package MRCE in R. This is for "Multivariate regression with covariance estimation".

$\endgroup$
2
  • 1
    $\begingroup$ Good suggestion! Doesn't seem to include observation weights, though. The weights ("lambdas" in the documentation), are for the independent variables... $\endgroup$
    – cmo
    Mar 15, 2013 at 20:05
  • $\begingroup$ The package suggested in this answer is now archived. $\endgroup$
    – EdM
    Aug 11, 2020 at 18:31
1
$\begingroup$

Case weights in a multivariate (multiple-outcome) regression don't have the straightforward meaning that they have in weighted least squares with a single outcome variable. Then each weight ideally represents the inverse of the variance of the corresponding outcome value, with error variances independent among cases. In a multivariate regression such an interpretation of a case weight would implicitly assume that all outcomes had the same relative variances from case to case. Also, a major reason for multivariate regression is to estimate the covariances among outcome values.

A work-around would be to take advantage of how, with a single outcome, a data transformation followed by OLS provides the same regression coefficients as weighted least squares. If you pre-multiply each of the design matrix and the outcome vector by the diagonal matrix of the square roots of the case weights, then OLS gives the same result as weighted least squares. As the regression coefficients returned by multivariate regressions are the same as those produced by regressions with each of the outcome variables individually, just extend that to pre-multiplying the outcome matrix--if you are willing to accept the consequences of any inapplicability of case weights to a multivariate regression. Transform the data first, then do the mulitivariate regression.

Despite the fear raised by the OP, lm() handles unweighted multivariate regressions quite well. It produces "mlm" objects that contain all the information needed for standard multivariate inference. See Fox and Weisberg. The R stats package simply (and I expect for reasons noted above) refuses to process a weighted multivariate regression beyond the estimation of the coefficients.

$\endgroup$
0
$\begingroup$

This is an old post, but the OP is factually wrong in claiming R doesn't do multi-variate regression.

The documentation states "If response is a matrix a linear model is fitted separately by least-squares to each column of the matrix." The key thing here is RESPONSE is a matrix. That is the Y is a matrix, then R fits ncol(Y) separate models to the same X: Y(i) ~ X.

$\endgroup$
2
  • 4
    $\begingroup$ I'm glad you're not downvoting, because the misunderstanding might be on your part: while multiple regression deals with multiple independent variables, multivariate regression concerns a vector-valued dependent (response) variable. It is not the same as a set of independently fitted multiple regressions to the components of the response. $\endgroup$
    – whuber
    Nov 11, 2013 at 21:16
  • $\begingroup$ This answer is correct, so far as it goes. That lm()` in R estimates regression coefficients response-by-response does not matter; as the coefficient estimates are the same. For unweighted multivariate regressions, which produce "mlm" objects, the standard methods for true multivariate inference are available. "Residual sums of squares and products" are provided by SSD(), estVar() provides the matrix of correlated residuals, and vcov() provides the appropriate Kronecker product for the coefficient-estimate covariances. It does not, however, handle weighted multivariate regressions. $\endgroup$
    – EdM
    Aug 11, 2020 at 12:12

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.