2
$\begingroup$

I have a dataset concerning patent applications from the years 1998-2018 and have two linear models fittet, one from 1998-2011, one from 2012-2018. I got two statistically significant models (both $p < 0.05$) but now I want to compare them to each other. Ideally, I would get the result that they are indeed different from each other and there is a significant difference between the two regression models pre-2011 and post-2011. The output for the regressions looks like this:

lm(formula = n ~ earliest_filing_year, data = hesc_appl_US[hesc_appl_US$earliest_filing_year > 
    1997 & hesc_appl_US$earliest_filing_year < 2012, ])

Residuals:
    Min      1Q  Median      3Q     Max 
-24.218 -14.371  -4.941  17.410  31.059 

Coefficients:
                      Estimate Std. Error t value Pr(>|t|)  
(Intercept)          -6909.727   2417.660  -2.858   0.0144 *
earliest_filing_year     3.468      1.206   2.875   0.0139 *
---
Signif. codes:  0 ‘***’ 0.001 ‘**’ 0.01 ‘*’ 0.05 ‘.’ 0.1 ‘ ’ 1

Residual standard error: 18.19 on 12 degrees of freedom
Multiple R-squared:  0.4079,    Adjusted R-squared:  0.3586 
F-statistic: 8.268 on 1 and 12 DF,  p-value: 0.01395

and

lm(formula = n ~ earliest_filing_year, data = hesc_appl_US[hesc_appl_US$earliest_filing_year > 
2011 & hesc_appl_US$earliest_filing_year < 2018, ])

Residuals:
      1       2       3       4       5       6 
-17.238  17.190   6.619  -3.952   5.476  -8.095 

Coefficients:
                      Estimate Std. Error t value Pr(>|t|)  
(Intercept)          21044.524   6583.526   3.197   0.0330 *
earliest_filing_year   -10.429      3.268  -3.191   0.0332 *
---
Signif. codes:  0 ‘***’ 0.001 ‘**’ 0.01 ‘*’ 0.05 ‘.’ 0.1 ‘ ’ 1

Residual standard error: 13.67 on 4 degrees of freedom
Multiple R-squared:  0.718, Adjusted R-squared:  0.6475 
F-statistic: 10.18 on 1 and 4 DF,  p-value: 0.03318

So far, I have tried using a modified version of the anova function in R in the following way:

mod1 <- lm(n ~ earliest_filing_year * I(earliest_filing_year > 2011), data = hesc_appl_US)
mod0 <- lm( n ~ earliest_filing_year, data = hesc_appl_US)

which gave me:

Analysis of Variance Table

Model 1: n ~ earliest_filing_year
Model 2: n ~ earliest_filing_year * I(earliest_filing_year > 2011)
  Res.Df     RSS Df Sum of Sq      F    Pr(>F)    
1     29 11223.1                                  
2     27  5428.2  2    5794.9 14.412 5.512e-05 ***
---
Signif. codes:  0 ‘***’ 0.001 ‘**’ 0.01 ‘*’ 0.05 ‘.’ 0.1 ‘ ’ 1

Now, considering this output, can I claim that there was an effect in 2011 and that the regressions are significantly different from each other? I would like to think that the $Pr$($>F$) is the $p$-value here that tells me that the regressions differ. Am i correct in this assumption or did I misunderstand the output?

$\endgroup$
1
  • 1
    $\begingroup$ Your approach with an interacting dummy and a partial F test is reasonable. $\endgroup$ – Michael M Sep 2 '19 at 11:53
1
$\begingroup$

I think you want to combine them into a single dataset, with a "dummy" (indicator) variable for before/after which would be zero for 1998-2011 and unity for 2012-2018. Then that dummy variable tells you the "effect" of being post-2011. You could also potentially consider an interaction term with your predictor variable (or not). My biggest worry however is an invalidation of regression assumptions (in your model you assume errors are independent, but because you have a time series there may be some autocorrelation from one year to the next). A smaller worry is that you are spending degrees of freedom depending how you "chose" the cutoff at 2001.

Following the discussion, if you create a variable called (say) post_2011 with 0 if it is <= 2011 and 1 if > 2011. Then you need to do formula = n ~ earliest_filing_year + post_2011 (no interactions) or formula = n ~ earliest_filing_year * post_2011 (to include the interactions)

$\endgroup$
6
  • $\begingroup$ Dear @Paul Hewson, thank you for your response. Me being quite the newbie to statistics am not quite sure what you mean by that exactly. I already have all the data in one dataset and have made two separate regressions for pre- and post-2011. What do you mean by "unity", exactly? Thank you very much! $\endgroup$ – P.Weyh Sep 2 '19 at 12:03
  • $\begingroup$ The idea is to incorporate in your model a dummy variable that indicates if you are pre (variable=0) 2011 or post (variable=1) 2011. $\endgroup$ – Elliot Sep 2 '19 at 12:05
  • $\begingroup$ Oooh, I see. Thanks you for the clarifcation. So basically I get a column that is 0 for all years <= 2011 and 1 for >2011 right? But how would I go about integrating that in the anova function? Or do I need to go back to stackoverflow with that question? $\endgroup$ – P.Weyh Sep 2 '19 at 12:22
  • $\begingroup$ create a variable called (say) post_2011 with 0 if it is <= 2011 and 1 if > 2011. Then you need to do formula = n ~ earliest_filing_year + post_2011 (no interactions) or formula = n ~ earliest_filing_year * post_2011 (to include the interactions) $\endgroup$ – Paul Hewson Sep 2 '19 at 15:22
  • $\begingroup$ There might be an issue of different error variances for the two time periods. Not all software packages can accommodate for that although one can look at histograms of residuals to see if there's any major departure from the assumption of equal variances. (But as you point out, maybe accounting for the potential serial correlation might be more likely to cause issues such as underestimating the standard errors of the parameter estimators.) $\endgroup$ – JimB Sep 2 '19 at 15:30

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.