5
$\begingroup$

Very simple. I am looking for a package that does Multivariate Linear Regression with weights on the observations. Does anyone know of a package that does this? I am shocked that I have not been able to find any.

NOTE: R does NOT do multivariate regression. The lm() help page specifically states: "If response is a matrix a linear model is fitted separately by least-squares to each column of the matrix. " This means independent regression models for each response variable. Thus lm() does NOT do multivariate linear regression. It merely does several univariate linear regressions for convenience.

Improve this question
$\endgroup$
1
  • 1
    $\begingroup$ Although it is correct that lm() does not handle weighted multivariate regression, it does do unweighted multivariate regression properly. Fitting a least-squares estimate separately to each column of the response matrix provides the correct coefficient estimates. The "mlm" objects returned by lm() for models with response matrices contain the information needed for true multivariate inference. See Fox and Weinberg, and my further comments on an answer below. $\endgroup$
    – EdM
    Commented Aug 11, 2020 at 12:19

3 Answers 3

Reset to default
2
$\begingroup$

Try package MRCE in R. This is for "Multivariate regression with covariance estimation".

Improve this answer
$\endgroup$
2
  • 1
    $\begingroup$ Good suggestion! Doesn't seem to include observation weights, though. The weights ("lambdas" in the documentation), are for the independent variables... $\endgroup$
    – cmo
    Commented Mar 15, 2013 at 20:05
  • $\begingroup$ The package suggested in this answer is now archived. $\endgroup$
    – EdM
    Commented Aug 11, 2020 at 18:31
1
$\begingroup$

Case weights in a multivariate (multiple-outcome) regression don't have the straightforward meaning that they have in weighted least squares with a single outcome variable. Then each weight ideally represents the inverse of the variance of the corresponding outcome value, with error variances independent among cases. In a multivariate regression such an interpretation of a case weight would implicitly assume that all outcomes had the same relative variances from case to case. Also, a major reason for multivariate regression is to estimate the covariances among outcome values.

A work-around would be to take advantage of how, with a single outcome, a data transformation followed by OLS provides the same regression coefficients as weighted least squares. If you pre-multiply each of the design matrix and the outcome vector by the diagonal matrix of the square roots of the case weights, then OLS gives the same result as weighted least squares. As the regression coefficients returned by multivariate regressions are the same as those produced by regressions with each of the outcome variables individually, just extend that to pre-multiplying the outcome matrix--if you are willing to accept the consequences of any inapplicability of case weights to a multivariate regression. Transform the data first, then do the mulitivariate regression.

Despite the fear raised by the OP, lm() handles unweighted multivariate regressions quite well. It produces "mlm" objects that contain all the information needed for standard multivariate inference. See Fox and Weisberg. The R stats package simply (and I expect for reasons noted above) refuses to process a weighted multivariate regression beyond the estimation of the coefficients.

Improve this answer
$\endgroup$
0
$\begingroup$

This is an old post, but the OP is factually wrong in claiming R doesn't do multi-variate regression.

The documentation states "If response is a matrix a linear model is fitted separately by least-squares to each column of the matrix." The key thing here is RESPONSE is a matrix. That is the Y is a matrix, then R fits ncol(Y) separate models to the same X: Y(i) ~ X.

Improve this answer
$\endgroup$
5
  • 4
    $\begingroup$ I'm glad you're not downvoting, because the misunderstanding might be on your part: while multiple regression deals with multiple independent variables, multivariate regression concerns a vector-valued dependent (response) variable. It is not the same as a set of independently fitted multiple regressions to the components of the response. $\endgroup$
    – whuber ♦
    Commented Nov 11, 2013 at 21:16
  • $\begingroup$ This answer is correct, so far as it goes. That lm()` in R estimates regression coefficients response-by-response does not matter; as the coefficient estimates are the same. For unweighted multivariate regressions, which produce "mlm" objects, the standard methods for true multivariate inference are available. "Residual sums of squares and products" are provided by SSD(), estVar() provides the matrix of correlated residuals, and vcov() provides the appropriate Kronecker product for the coefficient-estimate covariances. It does not, however, handle weighted multivariate regressions. $\endgroup$
    – EdM
    Commented Aug 11, 2020 at 12:12
  • $\begingroup$ @EdM In my research on inference for MLM, I came across your comment. I indeed want to do inference with weighted MLM and noticed this problem that R cannot do that. I even asked a question on that. Is the effort involved too great, or is it simply not possible? Do you perhaps know of a way to determine the vcov of an MLM in R? I need it to calculate HC0 standard errors of the MLM. $\endgroup$
    – jay.sf
    Commented Jul 2, 2023 at 10:10
  • $\begingroup$ @jay.sf my answer explains why I think that weighted multivariate analyses aren't supported: as case weights are typically based on an estimate of residual variance, having multiple outcomes would tend to mean different case weights for each outcome. If you have another reliable weighting method, my answer also includes a suggestion for a data transformation that could accomplish what you want. $\endgroup$
    – EdM
    Commented Jul 2, 2023 at 13:52
  • $\begingroup$ @jay.sf I've now illustrated the suggested data transformation as an answer to your question. $\endgroup$
    – EdM
    Commented Jul 2, 2023 at 14:45

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.