6
$\begingroup$

Suppose, we want to regress $y$ on $x_1$ and $x_2$, i.e.

$$ y = \alpha + \beta_1 x_1 + \beta_2 x_2 + \varepsilon \hspace{1cm} (1)$$

Is it, in principle, possible that simultaneously:

  • $\beta_1$ is statistically significant but $\beta_2$ is not while
  • $\beta_1$ statistically equals to $\beta_2$

?

$\endgroup$
2
  • $\begingroup$ What do you mean by "statistically equal"? If you mean $\hat{\beta}_1$ and $\hat{\beta}_2$ have the same estimated distribution, then no, it's not possible. If you mean that the point estimates $\hat{\beta}_1 = \hat{\beta}_2$ then yes it's possible. $\endgroup$ Commented Jun 30, 2015 at 17:35
  • $\begingroup$ I mean statistically equal according to some test, that means point estimates. I understand that running the same regression on different simulated data I can get this result, but is it possible to get it systematically? If so, how should I do that? $\endgroup$
    – Alekz112
    Commented Jun 30, 2015 at 18:42

1 Answer 1

8
$\begingroup$

Yes. This answer interprets the question in the following way:

  • $\beta_1$ is significantly different from zero in the full model

    $$y = \alpha + \beta_1 x_1 + \beta_2 x_2 + \varepsilon$$

  • $\beta_2$ is not significantly different from zero in the full model.

  • Either (a) $\beta_1=\beta_2$ or (b) a test of $H_0:\beta_1=\beta_2$ is not significant. The latter is equivalent to the full model not being significantly better than the reduced model

    $$y = \alpha + \beta(x_1 + x_2) + \varepsilon.$$

Intuitively, $y$ must have a detectable linear relationship with $x_1$ but not with $x_2$, even though the coefficients ("slopes") of those relationships are the same. This could happen when the spread of $x_1$ in the data is substantially greater than the spread of $x_2$: the wider spread of $x_1$ will induce greater changes in $y$, even when $\beta_1 \approx \beta_2$, making $\beta_1$ more readily detectable than $\beta_2$.

To illustrate, I played around with (a) the amount of data $n$ and (b) the variance of $\varepsilon$ to produce this phenomenon. The data are

$$(x_1, x_2, y) = ((1, 2, \ldots, 2n), (1,\ldots,1,-1,\ldots,-1), x_1+x_2+\varepsilon)$$

where $\varepsilon$ are independently and identically distributed with a mean of zero and standard deviation of $3$. As $n$ grows, $x_1$ becomes more spread out (from $1$ through $2n$) while $x_2$ is confined to the interval $[-1,1]$. The true underlying relationship is $\alpha=0, \beta_1=\beta_2=1$.

The following is R code to generate this example.

n <- 12
x1 <- 1:(2*n)
x2 <- c(rep(-1,n), rep(1,n))
set.seed(17)
y <- x1 + x2 + rnorm(2*n, sd=3)
  1. Here is the fit of the full model.

    > summary(fit.full <- lm(y ~ x1+x2))
    
    Coefficients:
                Estimate Std. Error t value Pr(>|t|)    
    (Intercept)  -0.5223     1.8358  -0.284    0.779    
    x1            1.1400     0.1416   8.053 7.41e-08 ***
    x2            0.4886     0.9800   0.499    0.623    
    

    $\beta_1$ is significant at any reasonable threshold ($p$ is essentially zero), while, $\beta_2$ is not significant at any reasonable threshold ($p=0.623$).

  2. The full model is not a significant improvement over the full model ($p = 0.5618$):

    >fit.partial <- lm(y ~ I(x1+x2))
    >anova(fit.partial, fit.full)
    
    Analysis of Variance Table
    
    Model 1: y ~ I(x1 + x2)
    Model 2: y ~ x1 + x2
      Res.Df    RSS Df Sum of Sq      F Pr(>F)
    1     22 122.36                           
    2     21 120.37  1    1.9924 0.3476 0.5618
    
$\endgroup$
10
  • $\begingroup$ Thanks, William, for the extensive and reasonable arguments! :) A short comment: x2 <- c(rep(-1,n), rep(1,n)) will not give you a sequence of (1, -1, 1, -1, ...) but rather (-1, -1,..., 1, 1). And if it is the former case, x2 turns to be significant immediately. Anyway, I like your logic. Basically, it means that if I set x1 <- rnorm(2*n, sd=5), x2 <- rnorm(2*n, sd=0.1), and y <- 1 + x1 + x2 + rnorm(2*n, sd=3), I should also get the very same result, as x1 is much more spread out than x2. And I do, but it depends on seed. Should we expect it? $\endgroup$
    – Alekz112
    Commented Jul 1, 2015 at 11:02
  • $\begingroup$ Thank you for pointing out my incorrect description of $X_2$--it was a holdover from an earlier version of this answer. (I changed it in order to make the standardized variables orthogonal.) The result should depend on the random seed, especially because the sample size is relatively small (just 24 observations). Logically, all that matters is that there exist some dataset with these characteristics, but it's more satisfying to show how one might arise naturally in a realistic regression setting, as done here. $\endgroup$
    – whuber
    Commented Jul 1, 2015 at 13:03
  • 1
    $\begingroup$ +1, @Alekz112, Andrew Gelman and Hal Stern had an American Statistician article about this exact question, see The difference between “significant” and “not significant” is not itself statistically significant. $\endgroup$
    – Andy W
    Commented Jul 1, 2015 at 13:17
  • $\begingroup$ @whuber, to clear it out: do we care about orthogonality because it can lead to multicollinearity and, thus, to less efficient estimates, which could bias our conclusions about significance? Or is there another reason? Apart from the seed, the result depends on the sample size as well, but of course, you're right: it is sufficient to find just one case. And finally, after reconsidering my question, I would start the answer something like this: yes, it is possible, if the confidence interval of $\hat{\beta_2}$ includes both 0 and $\hat{\beta_1}$. $\endgroup$
    – Alekz112
    Commented Jul 3, 2015 at 10:04
  • $\begingroup$ @AndyW, that is a really interesting article, it made some things in my head clear, thanks a lot for the reference! $\endgroup$
    – Alekz112
    Commented Jul 3, 2015 at 10:05

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.