3
$\begingroup$

My question is like this.

The regression specification is:

$y=\delta_{1}D_{1}+\delta_{2}D_{2}+\delta_{3}D_{3}+\beta_{1}x_{1}+\beta_{2}x_{2}+\beta_{3}x_{3}+\epsilon$

where $D_{i}$ denotes dummy for each one of three categories, $x_{i}$ interaction between $D_{i}$ and independent variable $x$.

I want to test $b_{1}=b_{2}=b_{3}$. $b_{i}$ is the estimated coefficient of $x_{i}$. After test, I will combine interaction terms for the categories not showing significantly different coefficients.

Each category has different sample size. Category 1 have much more samples than category 2, and category 2 more than category 3.

Estimates show that $b_{1}$ and $b_{2}$ is significant (from zero), but $b_{3}$ not. I think the small sample size of category 3 could explain insignificance of $b_{3}$.

My question: Is it meaningful (or right) to do joint test $b_{1}=b_{2}=b_{3}$ when $b_{3}$ is insignificant?

If the test $b_{1}=b_{2}=b_{3}$ can not be rejected at e.g 5% level but $b_{1}$ and $b_{3}$ is significantly different, could I say no significant difference among $x_{1}$, $x_{2}$ and $x_{3}$?

Improve this question
$\endgroup$

1 Answer 1

Reset to default
4
$\begingroup$

Re: your first question "Is it meaningful (or right) to do joint test $b_1=b_2=b_3$ when $b_3$ is insignificant?"

The null hypotheses are different - in one case you're testing a heterogeneity of means and in another you're testing whether a particular coefficient is 0. The fact that two coefficients are not significantly different from 0 does not preclude them from being different from each other. For example, if $\hat{\beta}_{1} = -1$ and $\hat{\beta}_{2} = 1$ and ${\rm se}(\hat{\beta}_{1}) = {\rm se}(\hat{\beta}_{2})=1$. In that case, $H_0 : \beta_{1}=\beta_{2}$ would be rejected (using the normal approximation) at the 5% level but $H_0 : \beta_{1}=0$ and $H_0 : \beta_{2}=0$ both would not.

Re: your second question - "If the test $b_1=b_2=b_3$ can not be rejected at e.g 5% level but $b_1$ and $b_3$ is significantly different, could I say no significant difference among $x_1$, $x_2$ and $x_3$?" - again the null hypotheses are different. In one case, you're testing a heterogeneity of means and in the second case you're testing for a pairwise difference. The two tests need not agree. This is a frequently discussed topic related to how ANOVA is different from pairwise tests and there are many threads on the subject. For example, see my answer here: In matlab, results from anova1 and multcompare disagree?

Improve this answer
$\endgroup$
3
  • $\begingroup$ Actually the reason to test $b_{1}=b_{2}=b_{3}$ is to combine groups. If the joint test is significantly rejected, I will separate these three categories i.e. regression including three interaction terms. But if the joint test can not be rejected, I will put them together i.e. there is not interaction term $x_{i}$ but only using independent variable $x$. You point out the difference between pairwise test and ANOVA. That is correct. But I don't know whether it is correct to put three categories together when $b_{1}=b_{2}=b_{3}$ can not be rejected but $b_{1}=b_{3}$ is rejected. $\endgroup$
    – Yang
    Commented Mar 31, 2012 at 9:50
  • $\begingroup$ Macro. I don't have enough reputation to upvote our answer. So I only choose accept. Thank you! and hope you to give me more help.@Macro $\endgroup$
    – Yang
    Commented Mar 31, 2012 at 9:59
  • 1
    $\begingroup$ (+1) Also to note, there is a paper by Andrew Gelman and Hal Stern that discusses this issue as well, The difference between “significant” and “not significant” is not itself statistically significant. $\endgroup$
    – Andy W
    Commented Mar 31, 2012 at 17:02

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.