I have a set of models that are the result of a multiple linear regression. I would like to calculate the power for each of these models. I found this tutorial on calculating the power using G*Power. Since the data has already been collected, I am using the Post Hoc type of Power Analysis, and I was hoping to use the "Determine" button to automatically calculate the effect size. However, I see that one need to provide a partial R2, but I would think that for a multiple linear regression (which I think would have multiple partial R2 values), it would make more sense to provide the R2 for the resulting model. Am I misunderstanding the difference between R2 and partial R2? If so, how do I obtain the partial R2 to use in G*Power?
$\begingroup$
$\endgroup$
5
asked Apr 14, 2013 at 18:08
-
3$\begingroup$ It's questionable whether one should do post-hoc power analysis at all. See e.g this article. This refers to post-hoc as in after the analysis has been done, not just after the sample has been collected. $\endgroup$– Peter FlomCommented Apr 14, 2013 at 18:13
-
$\begingroup$ Interesting. Someone else has already done analysis on this data, where we did achieve significant results. However, someone else was questioning whether we had enough samples to validly perform a multiple linear regression. Does the presence of significance indicate that we do have enough samples, or is there another way to determine this? $\endgroup$– cryptic_starCommented Apr 14, 2013 at 18:17
-
1$\begingroup$ Are they worried about power or over-fitting? If your result was significant, you had enough power to detect the effect size that you found. But if your N was less than about 10 times the number of variables, that can be problematic. $\endgroup$– Peter FlomCommented Apr 14, 2013 at 18:40
-
$\begingroup$ Our N is greater than 10 times the number of variables for some models, and less than 10 times the number of variables for other models. What does having an insufficient N indicate, even if you do get significance? $\endgroup$– cryptic_starCommented Apr 14, 2013 at 18:52
-
$\begingroup$ That would indicate possible overfitting of the data. To give an extreme example, if you have N-1 independent variables, you will get a perfect fit even if all the data is random. $\endgroup$– Peter FlomCommented Apr 14, 2013 at 20:44
Add a comment
|
1 Answer
$\begingroup$
$\endgroup$
1
In GPower, you do a power for an R2 in multiple regression by doing the partial R2 with no predictors in the baseline model.
To do this, set the total number of predictors to 1, and the number of tested predictors to 1. You're then testing the model against an intercept only model, with an R2 of zero.
(You can always think of regression models in this way - you're testing against no predictors, and looking at the change in R2.)
answered Apr 15, 2013 at 4:35
-
$\begingroup$ Thanks for the explanation. I had never heard of regression as being a change in R2. $\endgroup$ Commented Apr 15, 2013 at 11:14