I am confused.

How to interpret the following results? A variable has a highly significant effect in a model one, however, a second model without the variable has similar (non-different) R2?

Thus, the variable has an effect on dependent variable, but this does not explain much of the variance?

  • $\begingroup$ Could you please provide additional information about your dataset? Moreover, it would be helpful if you could add your script as well. $\endgroup$ – bilibraker Jun 30 at 9:33
  • $\begingroup$ What do you mean by “non-different”? Also, how many observations do you have? Small differences can be highly significant when large sample sizes make your hypothesis test extremely sensitive. $\endgroup$ – Dave Jun 30 at 10:23
  • $\begingroup$ With non-different, I mean that there are overlapping confidence intervals. R2 model with variable: 0.15 (CI: 0.11-0.20). R2 model without the variable: 0.13 (CI: 0.09-0.20). These are retrieved using the function bayes_R2(modelname). Plus yes, I have large dataset, about 5000 subjects. $\endgroup$ – st4co4 Jun 30 at 12:06
  • $\begingroup$ Please edit your question to add the information in your comment. $\endgroup$ – Peter Flom 2 days ago

There might be a strong correlation between variables.

For example, let's consider correlated variables, $X_1$ and $X_2$ with relationship $X_1 = k \cdot X_2$.

If you train your model including two variables with linear regression, you would get this linear equation: $$ y = \tilde{\alpha}_1 X_1 + \tilde{\alpha}_2 X_2 + \tilde{\varepsilon} $$

where $\tilde{\varepsilon}$ is residuals.

With the formula, $X_1 = k \cdot X_2$, the equation is converted into this form: $$ \begin{align} y & = \tilde{\alpha}_1 \cdot k \cdot X_2 + \tilde{\alpha}_2 X_2 + \tilde{\varepsilon} \\ & = (\tilde{\alpha}_1 k + \tilde{\alpha}_2)X_2 + \tilde{\varepsilon} \end{align} $$

What this equation means that even if you train your model only with the $X_2$, you get the same residuals. And if you have the same residuals, you would get the same $R^2$.

| cite | improve this answer | |

Not the answer you're looking for? Browse other questions tagged or ask your own question.