0
$\begingroup$

I want to show that the variable $X$ is significant.
In model 1, I use financial statements variables, other macroeconomic variables and $X$.
Here, $X$ is siginificant at a 10% level and $R^2$ is 0.731.
Since financial statements could have time lag, I have to check that effect so I made adjusted model 2.
In here $X$ is insignificant but $R^2$ is 0.721.
So, I can say that "since $R^2$ is decreased, model 1 is better and $X$ is significant variable"?

$\endgroup$
3
  • 1
    $\begingroup$ It's difficult to know what you are seeking here. If $X$ is insignificant (at whatever level), it is insignificant. $R^2$ quantifies a different question. Change in $R^2$ has nothing to do with a significance calculation; the model doesn't remember what you did previously. $\endgroup$
    – Nick Cox
    Nov 27, 2013 at 13:12
  • $\begingroup$ @NickCox If the models are nested, isn't looking at the change in $R^2$ equivalent to the partial F-test? $\endgroup$
    – jsk
    May 15, 2014 at 1:30
  • $\begingroup$ @jsk Fair point, but only in the sense that looking at the difference between means is "equivalent to" a t-test. One is a descriptive calculation, the other a specific inference. $\endgroup$
    – Nick Cox
    May 15, 2014 at 18:20

1 Answer 1

3
$\begingroup$

Without more information about what you're trying to achieve, I'm just going to direct my answer to your apparent confusion regarding the difference between p-value and R-squared.

Significance is determined by your p-value. It appears that you have set your alpha to 0.10 (this is somewhat unusual - do you have justification for setting it this high?) so that means that any variable with a p-value less than 0.10 is going to show up as "significant". P-value is just a measure of the probability of getting a result as great or greater than what you observed, if the null were true. So for instance if your X variable actually had a p-value of 0.03, then 3% of the time running whatever data-collection you did you would "find" an X value that is as great as what you've observed, or greater (farther away from 0), even if the true value of X in real life IS zero. It's basically the risk you're taking of incorrectly concluding something (and with your significance level of .10, you've agreed to accept up to a 10% risk of a wrong conclusion).

R-squared measures the percent of variation in Y explained by variation in X (or combination of Xs - which is why we generally use adjusted R-squared so that throwing more unrelated variables in doesn't artificially and accidentally explain some of the variation when it really has nothing to do with Y). A high R-squared doesn't necessarily mean something is good, and a low one doesn't mean it is bad. In fact, a high R-squared with insignificant variables in the model doesn't tell you much at all. But a low R-squared with a well-built, significant model can tell you that you've discerned something interesting, even if it doesn't explain the whole picture. For another example, I am currently trying to figure out what environmental factors affect bacterial growth. I've built a model with some temperatures and salinity, and the R-squared is 0.19. 19% of the variation in bacterial growth explained by variation in temperature and salinity seems pretty good to me since we haven't yet included a lot of other possible environmental variables, and bacterial growth is complicated!

As a final (but most important) point, it's a red flag for someone to want to "show" that a particular variable is significant. It either is, or isn't - just as an independent variable either does or does not impact your dependent variable in real life. Manipulating your model to include a variety of conditions in order to achieve significance on one variable or another is heading into dangerous territory and I would avoid it at all cost. If you have an interest to keep reading, the first answer to this question explains well the problems with blindly changing things in a regression: Algorithms for automatic model selection

$\endgroup$

Not the answer you're looking for? Browse other questions tagged or ask your own question.