4
$\begingroup$

I did a (multi)linear regression of Y on a collection $X_1, X_2,..,X_k$ of variables.

Some of the coefficients of the $X_i $ were very close to 1. Can I conclude that Y and $X_i$are , in a sense, equivalent to each other (adjusted $R^2$ was also pretty high)?

The standard interpretation of the coefficient $b_i$ of $X_i$ I am aware of is that a change of 1 unit in $X_i$ results in a change of $b_i$ units in Y. Now, if $b_i$ is close to 1, this says a change of 1 unit of $X_i$ results in a change of 1 unit in Y, so that, in a sense, $X_i$ does not provide any information about Y, it is in a sense equal to Y. Is this correct? If so, is there a way of making this more precise?

Improve this question
$\endgroup$
9
  • 2
    $\begingroup$ If you have multiple variables and all coefficients are 1, this implies Y is best approximated by the sum of all the variables, not by any individual one. $\endgroup$
    – josliber
    Commented Jul 10, 2016 at 1:43
  • $\begingroup$ @josliber: but what if the restricted regression coefficients Y|X_i , i.e., restricting to a single regressor and , are also close to 1, and each having a high R^2 ? Doesn't that imply each X_i itself is somehow equivalent to X? $\endgroup$
    – MSIS
    Commented Jul 10, 2016 at 1:55
  • $\begingroup$ In simple linear regression if you have an intercept of 0, coefficient of 1, and high R^2 then yes the outcome is well approximated by the single variable. However, your question is written about multivariate analysis, not simple linear regression with a single variable. If you are asking about simple linear regression, please update your question accordingly. $\endgroup$
    – josliber
    Commented Jul 10, 2016 at 1:59
  • $\begingroup$ No, I meant Y|X_i , where the regression in question is Y|X_1, X_2,..,X_k , i.e., I am referring to a multiple regression restricted to a single variable and getting the given results: high R^2 and a coefficient close to 1. $\endgroup$
    – MSIS
    Commented Jul 10, 2016 at 2:34
  • $\begingroup$ What do you mean when you say "multiple regression restricted to a single variable"? Do you mean you are noticing that one of the variables has coefficient 1 from among all the variables in a multivariate model, or are you fitting a new linear regression model with just that single variable? $\endgroup$
    – josliber
    Commented Jul 10, 2016 at 2:37

1 Answer 1

Reset to default
6
$\begingroup$

No, just because a coefficient value is close to 1 in multiple linear regression does not imply that the variable is almost equal to the outcome. Consider a simple example:

(dat <- data.frame(y=c(1.1001, 2.1002, 3.0997, 4.0001, 4.9999), x1=c(.1, .1, .1, 0, 0), x2=1:5))
#        y  x1 x2
# 1 1.1001 0.1  1
# 2 2.1002 0.1  2
# 3 3.0997 0.1  3
# 4 4.0001 0.0  4
# 5 4.9999 0.0  5

In this dataset, the outcome variable y is basically equal to the sum of the two independent variables x1 and x2; as expected the linear regression has an excellent R^2 and both coefficients are close to 1:

summary(lm(y~., data=dat))
# Coefficients:
#              Estimate Std. Error  t value Pr(>|t|)    
# (Intercept) 0.0009000  0.0005079    1.772    0.218    
# x1          0.9950000  0.0031623  314.647 1.01e-05 ***
# x2          0.9998000  0.0001095 9126.884 1.20e-08 ***
# ---
# Signif. codes:  0 ‘***’ 0.001 ‘**’ 0.01 ‘*’ 0.05 ‘.’ 0.1 ‘ ’ 1
# 
# Residual standard error: 0.0001732 on 2 degrees of freedom
# Multiple R-squared:      1,   Adjusted R-squared:      1 
# F-statistic: 1.569e+08 on 2 and 2 DF,  p-value: 6.376e-09

However, not only is variable x1 not "in a sense equal" to y, but it is actually negatively correlated with y (correlation coefficient -0.857).

On the other hand, if you fit a regression model using a single variable and it has coefficient close to 1, intercept close to 0, and high R^2, then indeed you can conclude that the variable is very similar to the outcome.

Improve this answer
$\endgroup$
0

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.