I want to show that the variable $X$ is significant.
In model 1, I use financial statements variables, other macroeconomic variables and $X$.
Here, $X$ is siginificant at a 10% level and $R^2$ is 0.731.
Since financial statements could have time lag, I have to check that effect so I made adjusted model 2.
In here $X$ is insignificant but $R^2$ is 0.721.
So, I can say that "since $R^2$ is decreased, model 1 is better and $X$ is significant variable"?
-
1$\begingroup$ It's difficult to know what you are seeking here. If $X$ is insignificant (at whatever level), it is insignificant. $R^2$ quantifies a different question. Change in $R^2$ has nothing to do with a significance calculation; the model doesn't remember what you did previously. $\endgroup$– Nick CoxNov 27, 2013 at 13:12
-
$\begingroup$ @NickCox If the models are nested, isn't looking at the change in $R^2$ equivalent to the partial F-test? $\endgroup$– jskMay 15, 2014 at 1:30
-
$\begingroup$ @jsk Fair point, but only in the sense that looking at the difference between means is "equivalent to" a t-test. One is a descriptive calculation, the other a specific inference. $\endgroup$– Nick CoxMay 15, 2014 at 18:20
1 Answer
Without more information about what you're trying to achieve, I'm just going to direct my answer to your apparent confusion regarding the difference between p-value and R-squared.
Significance is determined by your p-value. It appears that you have set your alpha to 0.10 (this is somewhat unusual - do you have justification for setting it this high?) so that means that any variable with a p-value less than 0.10 is going to show up as "significant". P-value is just a measure of the probability of getting a result as great or greater than what you observed, if the null were true. So for instance if your X variable actually had a p-value of 0.03, then 3% of the time running whatever data-collection you did you would "find" an X value that is as great as what you've observed, or greater (farther away from 0), even if the true value of X in real life IS zero. It's basically the risk you're taking of incorrectly concluding something (and with your significance level of .10, you've agreed to accept up to a 10% risk of a wrong conclusion).
R-squared measures the percent of variation in Y explained by variation in X (or combination of Xs - which is why we generally use adjusted R-squared so that throwing more unrelated variables in doesn't artificially and accidentally explain some of the variation when it really has nothing to do with Y). A high R-squared doesn't necessarily mean something is good, and a low one doesn't mean it is bad. In fact, a high R-squared with insignificant variables in the model doesn't tell you much at all. But a low R-squared with a well-built, significant model can tell you that you've discerned something interesting, even if it doesn't explain the whole picture. For another example, I am currently trying to figure out what environmental factors affect bacterial growth. I've built a model with some temperatures and salinity, and the R-squared is 0.19. 19% of the variation in bacterial growth explained by variation in temperature and salinity seems pretty good to me since we haven't yet included a lot of other possible environmental variables, and bacterial growth is complicated!
As a final (but most important) point, it's a red flag for someone to want to "show" that a particular variable is significant. It either is, or isn't - just as an independent variable either does or does not impact your dependent variable in real life. Manipulating your model to include a variety of conditions in order to achieve significance on one variable or another is heading into dangerous territory and I would avoid it at all cost. If you have an interest to keep reading, the first answer to this question explains well the problems with blindly changing things in a regression: Algorithms for automatic model selection