Take the 2-minute tour ×
Cross Validated is a question and answer site for people interested in statistics, machine learning, data analysis, data mining, and data visualization. It's 100% free, no registration required.

An basic question came to my mind while I was working on sample size calculation. I have some possible explanation and I would like to compare it to yours.

Let us say I would like to explain some response variable $Y$ from explanatory variables $A$ and $B$.

Further, let us assume that $B$ has no effect on $Y$; the corresponding regression coefficient being $0$.

Two possible models are:

$E(Y) = \mu + \beta_{1} A$,

and

$E(Y) = \mu + \beta_{1} A + \beta_{2} B$.

Which of these two should I choose in order to have more power to detect some effect of $A$.

What if $B$ has an effect ($\beta_{2} \neq 0$)?

share|improve this question
add comment

3 Answers

up vote 2 down vote accepted

In the case you know $B_2$ to be zero, and the covariance between $A$ and $B$ does not equal zero, the reduced model will have more power. The standard error of $B_1$ increases as a function of the shared variance between $A$ and the other independent variables in the regression (see this document I found while trying to google for a formula for the standard error of a regression coefficient). I italicize know because frequently you don't know A priori that the effect is zero (unless it is theoretically non-sense), and hence it would require being estimated to prevent a biased estimate of $B_1$ (although this is not directly pertinent to the power of the test). Does the power of a biased test make any sense?

I've seen this referred to as "effective sample size". I know the software such as G*Power 3 allows you to enter the $R^{2}$ from the auxiliary regression equation (that is in this example the regression of $A$ on $B$).

As I stated in reply to Henrik's answer, the loss of the 1 degree of freedom would be rather trivial in most practical sample sizes. The correlation between $A$ and $B$ is of much more pertinence. In the case that $A$ and $B$ aren't correlated this is all moot (except in regards to the degrees of freedom).

share|improve this answer
    
Thx very much. Give some time to study your answer before accepting it. –  ocram Aug 15 '11 at 16:46
    
@ocram, in addition to my answer, it also stands to say that the estimate of $B_1$ will differ between the full and reduced model if the $Cov(A,B) \neq 0$. Hence it isn't as simple as choosing the model with the most power in this case (either one of your models presented is correct and the other is wrong, or it makes little difference). But in general, what I state about the shared variance between independent variables and the estimate of the regression coefficient standard error is correct. –  Andy W Aug 15 '11 at 17:01
add comment

I think there will be some cases where the simpler model is more powerful, and some cases where the full model is more powerful. A couple of examples in R:

Example 1: Full model is more powerful

library(mvtnorm)
set.seed(1)
n <- 1000
X <- rmvnorm(n, sigma = matrix(c(1, 0.5, 0.5, 1), nrow=2))
A <- X[,1]
B <- X[,2]
Y <- 1 - A + 2*B + rnorm(n, 0, 1)
cor(A, Y)
[1] 0.03338526
# Simple correlation of A with Y will tend to be very low, so model
# with A alone will have low power

# These don't really demonstrate the power, but they give you an idea
summary(lm(Y ~ A + B))

Coefficients:
            Estimate Std. Error t value Pr(>|t|)    
(Intercept)  1.01624    0.03260   31.18   <2e-16 ***
A           -0.95187    0.03643  -26.13   <2e-16 ***
B            2.00990    0.03628   55.41   <2e-16 ***
---

summary(lm(Y ~ A))

Coefficients:
            Estimate Std. Error t value Pr(>|t|)    
(Intercept)  0.99436    0.06580  15.112   <2e-16 ***
A            0.06699    0.06348   1.055    0.292    

Example 2: Reduced model is more powerful

set.seed(1)
X <- rmvnorm(n, sigma = matrix(c(1, 0.5, 0.5, 1), nrow=2))
A <- X[,1]
B <- X[,2]
Y <- 1  + 0.01*A + 2*B + rnorm(n, 0, 1)
cor(A, Y)
[1] 0.4731269  # This correlation is made higher because of the correlation between A and B

# Again, these don't demonstrate the power, but they give you an idea
summary(lm(Y ~ A + B))

Coefficients:
            Estimate Std. Error t value Pr(>|t|)    
(Intercept)  1.01624    0.03260  31.176   <2e-16 ***
A            0.05813    0.03643   1.596    0.111    
B            2.00990    0.03628  55.407   <2e-16 ***

summary(lm(Y ~ A))

Coefficients:
            Estimate Std. Error t value Pr(>|t|)    
(Intercept)  0.99436    0.06580   15.11   <2e-16 ***
A            1.07699    0.06348   16.97   <2e-16 ***
# This Y ~ A model has higher power for an A effect than the Y ~ A + B model, but
# the effect it detects is not the "real" effect of A
share|improve this answer
1  
(+1) This answer would be more complete if you explain in text why if the correlation between A and B is higher the power of the test is lower (instead of burying it in code). –  Andy W Aug 15 '11 at 12:04
    
Upon a second reading, as stated this is not a good example. All you have demonstrated is bias, not power. –  Andy W Aug 15 '11 at 15:30
    
I don't think I've demonstrated power or bias. I was just providing examples that the question-asker could experiment with. I'm confident that a simulation would confirm the claims about power that I've made, but I also think that the power is not really relevant in the examples I've provided. The bias, as you mentioned, is more important in my opinion, so the full model should be preferred even though its "power" may be lower. –  mark999 Aug 15 '11 at 22:02
add comment

I would mainly argue with the degrees of freedom. In multiple regression the degrees of freedom are $N-k-1$. With $N$ being the overall sample size and $k$ the number of parameters.

Therefore, adding a parameter with no explanatory power will reduce the overall degrees of freedom. As the degrees of freedom are directly proportional to your power, adding another explanatory variable with no explanatory power will reduce your overall power.

See also this link (first one on google: "degree of freedom multiple regression")

share|improve this answer
    
This is true, but the loss due to degrees of freedom would be rather trivial in most situations I would think. The correlation between A and B is of more pertinence (as mark999 noted in his answer). –  Andy W Aug 15 '11 at 12:06
add comment

Your Answer

 
discard

By posting your answer, you agree to the privacy policy and terms of service.

Not the answer you're looking for? Browse other questions tagged or ask your own question.