2
$\begingroup$

DISCLAIMER: This question was sent to Stata list today, but so far nobody has answered.

NOTE: I use Stata here, but actually I don't think the question is software-specific.

Hi,

I would appreciate someone could provide me an answer to the following question:

I am estimating the following model:

. areg beta L.lev group#cL.lev i.year, absorb(group)

Group is a categorical variable: 1, 2 Lev is a continuous variable Year are year effects Group are group effects

Under that specification of the model, I get the following results:

    . areg beta L.lev group#cL.lev i.year, absorb(group)

Linear regression, absorbing indicators           Number of obs   =        285
                                                  F(  17,    266) =       4.04
                                                  Prob > F        =     0.0000
                                                  R-squared       =     0.3540
                                                  Adj R-squared   =     0.3103
                                                  Root MSE        =     0.3038

------------------------------------------------------------------------------
        beta |      Coef.   Std. Err.      t    P>|t|     [95% Conf. Interval]
-------------+----------------------------------------------------------------
         lev |
         L1. |    .059685    .014907     4.00   0.000     .0303342    .0890358
             |
group#cL.lev |
        2  |  -.0451045   .0178834    -2.52   0.012   -.0803155   -.0098936

What I would like to highlight from here, is the fact that the P value for group#cLlev 2 is significant at the 5% level.

If I run the same model in a different way (not including the variable Lev by itself), like this:

. areg beta group#cL.lev i.year, absorb(group)

Linear regression, absorbing indicators           Number of obs   =        285
                                                  F(  17,    266) =       4.04
                                                  Prob > F        =     0.0000
                                                  R-squared       =     0.3540
                                                  Adj R-squared   =     0.3103
                                                  Root MSE        =     0.3038

------------------------------------------------------------------------------
        beta |      Coef.   Std. Err.      t    P>|t|     [95% Conf. Interval]
-------------+----------------------------------------------------------------
group#cL.lev |
          1  |    .059685    .014907     4.00   0.000     .0303342    .0890358
          2  |   .0145805   .0114004     1.28   0.202    -.0078661     .037027

The P value for group 2 is not significant at the 5% level.

So the question is, which P-value should I consider as correct? In other words, is my parameter estimate significant or not?

(Please note that I get exactly the same parameter estimates from both methods, but the associated P-values and t-values change).

Improve this question
$\endgroup$

1 Answer 1

Reset to default
4
$\begingroup$

The two regression results are the same but leaving out the main effect (lev) changes the baseline for comparison of the two groups.

In the first regression lev L1. = 0.059685 is the coefficient for group 1 and group#cL.lev 2 = -0.0451045 is the deviation of group 2 from the baseline (group 1). This means that the coefficient for group 2 is -0.0451045 smaller than the coefficient of group 1.

In the second regression this is exactly what you get again just that the coefficient of group 2 is not expressed as the difference between the two groups. To see this go back to the first regression and subtract 0.059685 - 0.0451045 = 0.0145805, which is your group 2 coefficient in the second regression.

This means that the effect of lev on the outcome variable betafor group 2 is not significantly different from zero (p-value = 0.202) in the second regression. The first regression on the other hand tells you that the effect on group 2 compared to group 1 is -0.0451045 smaller and this difference is significant.

Improve this answer
$\endgroup$
5
  • $\begingroup$ Thank you Andy! One more, maybe redundant question, is: Should we trust the value of Lev 2? I mean, if I understand you correctly, what the model is telling us is that the coefficient estimate is not significant, and therefore we should not consider that value for neither of the two specifications of the model. In short, Lev 2 is not significant, never. The only significant (let's call it truth), is that it deviates from Lev 1 for the value reported. Am I right? $\endgroup$
    – Herman Haugland
    Commented Aug 21, 2013 at 17:39
  • 1
    $\begingroup$ Yes but it's not really a statement about trust though. It's more a tautology: if the effect of lev on beta is significant for group 1 but it is insignificant for group 2, it is logical that the difference of the effects between the two groups is also significant. In the end it depends what you are looking for - the effect for the two groups or the difference between the groups. You can get both from either regression. $\endgroup$
    – Andy
    Commented Aug 21, 2013 at 17:44
  • $\begingroup$ Please allow me a final question. I estimated the coefficient for Lev for the entire group of individuals (banks). So, instead of having groups 1 and 2, I just have one group with all the banks. The coefficient for Lev for this estimation is significant and smaller than for group 1 shown above. So the question is, if you had to decide which estimate to use, the one for all banks together or the one for just group 1 (which banks account for 86% of the sample, 56% of the total market), as reported above, which one would you pick? Would you base your decision on the value of the SE? $\endgroup$
    – Herman Haugland
    Commented Aug 21, 2013 at 18:42
  • 1
    $\begingroup$ It makes sense to differentiate between the two groups via the interaction as you did in your model above because the effect for the two groups is different. Basing model and variable selection on standard errors is generally not a good idea because it can be misleading. If you have some theoretical/economic intuition about why the effect differs between the two groups you will be able to make a stronger argument. $\endgroup$
    – Andy
    Commented Aug 21, 2013 at 19:00
  • $\begingroup$ Thank you very much Andy, your explanations are very clear. I appreciate you took your time to answer, thank you. $\endgroup$
    – Herman Haugland
    Commented Aug 21, 2013 at 21:45

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.