Probabilty Scale Versus Log Odds - Interaction Effect Size - Cross Validated most recent 30 from stats.stackexchange.com 2019-10-18T19:51:06Z https://stats.stackexchange.com/feeds/question/261222 https://creativecommons.org/licenses/by-sa/4.0/rdf https://stats.stackexchange.com/q/261222 3 Probabilty Scale Versus Log Odds - Interaction Effect Size B_Miner https://stats.stackexchange.com/users/2040 2017-02-10T23:36:49Z 2019-08-13T01:01:00Z <p>I am looking at doing some power analysis for a simple full factorial design using logistic regression as the method of analysis 2 main effects and their interaction. I am trying to discern the magnitude of the interaction through postulated probabilities.</p> <p>Specifically I have the following factors A and B, each with a low (-1) and high (1) level. Given a balanced experiment I believe I can average over the other level for the main effects and calculate the main effect of A as 0.11% and the main effect of B as 0.0934%. Slicing the simple effects of A I calculate the interaction as -0.08%.</p> <p>I am therefore thinking with the power analysis I should have almost as much power for the interaction term as the other main effects, especially B. It works out that the interaction term is tiny through the logistic regression.</p> <p>Is there any way to tell just from the probabilities if the interaction is "large" or "small"? </p> <pre><code>&gt; x_df&lt;-data.frame(A=c(-1,-1,1,1),B=c(-1,1,-1,1)) &gt; x_df A B 1 -1 -1 2 -1 1 3 1 -1 4 1 1 &gt; X&lt;-matrix(c(12,9988,5,9995,25,9975,10,9990),nrow = 4,ncol = 2,byrow = TRUE) &gt; X [,1] [,2] [1,] 12 9988 [2,] 5 9995 [3,] 25 9975 [4,] 10 9990 &gt; summary(glm(X~x_df\$A+x_df\$B+x_df\$A*x_df\$B, family = binomial)) Call: glm(formula = X ~ x_df\$A + x_df\$B + x_df\$A * x_df\$B, family = binomial) Deviance Residuals:  0 0 0 0 Coefficients: Estimate Std. Error z value Pr(&gt;|z|) (Intercept) -6.80509 0.16274 -41.817 &lt; 2e-16 *** x_df\$A 0.35723 0.16274 2.195 0.02815 * x_df\$B -0.44849 0.16274 -2.756 0.00585 ** x_df\$A:x_df\$B -0.01041 0.16274 -0.064 0.94901 --- Signif. codes: 0 ‘***’ 0.001 ‘**’ 0.01 ‘*’ 0.05 ‘.’ 0.1 ‘ ’ 1 (Dispersion parameter for binomial family taken to be 1) Null deviance: 1.5995e+01 on 3 degrees of freedom Residual deviance: 7.0046e-12 on 0 degrees of freedom AIC: 25.033 Number of Fisher Scoring iterations: 3 </code></pre> <p><a href="https://i.stack.imgur.com/MS9z6.png" rel="nofollow noreferrer"><img src="https://i.stack.imgur.com/MS9z6.png" alt="enter image description here"></a></p> https://stats.stackexchange.com/questions/261222/-/293806#293806 0 Answer by FelixST for Probabilty Scale Versus Log Odds - Interaction Effect Size FelixST https://stats.stackexchange.com/users/163818 2017-07-22T06:25:11Z 2017-07-22T07:06:15Z <p>I don't really know if you want a power analysis (which would be estimating the probability of having a significant result given that there is a true effect of a certain size, and which is not <em>really</em> a legitimate thing to do based on observed sample statistics).</p> <p>In terms of power, you <em>probably</em> had low power to detect the real value of the interaction if there is a real interaction effect - as you had a non-significant result - (emphasis on probably, as you may have just gotten very unlucky, there is no actual way of figuring this out without knowing the true population effect). The smaller the effect, the lower the power, all else being equal, so unless you expect that the interaction to be quite a lot larger than it is estimated to be here, you had super low power to detect an effect of that magnitude. It certainly wouldn't be the same as for the main effects. </p> <p>In fact, it would have had to have been massively overestimated (given that the magnitude here is accurate - not a safe assumption) to be statistically significant in your dataset. </p> <p>Effect sizes are notoriously difficult to interpret for interaction terms in log reg- the coefficients tell you how much of a change there is in the log odds, not in terms of percentages, and as it's an interaction, it can only be interpreted in the context of the scales of the two independent variables - you may want to have a look at <a href="https://stats.stackexchange.com/a/57052/163818">https://stats.stackexchange.com/a/57052/163818</a> for a primer on how to interpret logistic regression interactions. </p> https://stats.stackexchange.com/questions/261222/-/294768#294768 0 Answer by Horst Grünbusch for Probabilty Scale Versus Log Odds - Interaction Effect Size Horst Grünbusch https://stats.stackexchange.com/users/28705 2017-07-27T15:20:07Z 2017-07-27T15:20:07Z <p>There is a general rule of thumb: If you calculated a sample size of \$n\$ to test for the main effect, you'll need \$4n\$ for the interaction of the same magnitude and at the same power. See <a href="https://doi.org/10.1016/j.csda.2008.06.010" rel="nofollow noreferrer">Leon and Heo (CSDA, 2009)</a>. This applies to logistic regression as well. It can be worse for poorly balanced data. </p>