Compare linear models from different parts of the same dataset

Question

I have a dataset concerning patent applications from the years 1998-2018 and have two linear models fittet, one from 1998-2011, one from 2012-2018. I got two statistically significant models (both $p < 0.05$) but now I want to compare them to each other. Ideally, I would get the result that they are indeed different from each other and there is a significant difference between the two regression models pre-2011 and post-2011. The output for the regressions looks like this:

lm(formula = n ~ earliest_filing_year, data = hesc_appl_US[hesc_appl_US$earliest_filing_year > 
    1997 & hesc_appl_US$earliest_filing_year < 2012, ])

Residuals:
    Min      1Q  Median      3Q     Max 
-24.218 -14.371  -4.941  17.410  31.059 

Coefficients:
                      Estimate Std. Error t value Pr(>|t|)  
(Intercept)          -6909.727   2417.660  -2.858   0.0144 *
earliest_filing_year     3.468      1.206   2.875   0.0139 *
---
Signif. codes:  0 ‘***’ 0.001 ‘**’ 0.01 ‘*’ 0.05 ‘.’ 0.1 ‘ ’ 1

Residual standard error: 18.19 on 12 degrees of freedom
Multiple R-squared:  0.4079,    Adjusted R-squared:  0.3586 
F-statistic: 8.268 on 1 and 12 DF,  p-value: 0.01395

and

lm(formula = n ~ earliest_filing_year, data = hesc_appl_US[hesc_appl_US$earliest_filing_year > 
2011 & hesc_appl_US$earliest_filing_year < 2018, ])

Residuals:
      1       2       3       4       5       6 
-17.238  17.190   6.619  -3.952   5.476  -8.095 

Coefficients:
                      Estimate Std. Error t value Pr(>|t|)  
(Intercept)          21044.524   6583.526   3.197   0.0330 *
earliest_filing_year   -10.429      3.268  -3.191   0.0332 *
---
Signif. codes:  0 ‘***’ 0.001 ‘**’ 0.01 ‘*’ 0.05 ‘.’ 0.1 ‘ ’ 1

Residual standard error: 13.67 on 4 degrees of freedom
Multiple R-squared:  0.718, Adjusted R-squared:  0.6475 
F-statistic: 10.18 on 1 and 4 DF,  p-value: 0.03318

So far, I have tried using a modified version of the anova function in R in the following way:

mod1 <- lm(n ~ earliest_filing_year * I(earliest_filing_year > 2011), data = hesc_appl_US)
mod0 <- lm( n ~ earliest_filing_year, data = hesc_appl_US)

which gave me:

Analysis of Variance Table

Model 1: n ~ earliest_filing_year
Model 2: n ~ earliest_filing_year * I(earliest_filing_year > 2011)
  Res.Df     RSS Df Sum of Sq      F    Pr(>F)    
1     29 11223.1                                  
2     27  5428.2  2    5794.9 14.412 5.512e-05 ***
---
Signif. codes:  0 ‘***’ 0.001 ‘**’ 0.01 ‘*’ 0.05 ‘.’ 0.1 ‘ ’ 1

Now, considering this output, can I claim that there was an effect in 2011 and that the regressions are significantly different from each other? I would like to think that the $Pr$($>F$) is the $p$-value here that tells me that the regressions differ. Am i correct in this assumption or did I misunderstand the output?

Answer 1

I think you want to combine them into a single dataset, with a "dummy" (indicator) variable for before/after which would be zero for 1998-2011 and unity for 2012-2018. Then that dummy variable tells you the "effect" of being post-2011. You could also potentially consider an interaction term with your predictor variable (or not). My biggest worry however is an invalidation of regression assumptions (in your model you assume errors are independent, but because you have a time series there may be some autocorrelation from one year to the next). A smaller worry is that you are spending degrees of freedom depending how you "chose" the cutoff at 2001.

Following the discussion, if you create a variable called (say) post_2011 with 0 if it is <= 2011 and 1 if > 2011. Then you need to do formula = n ~ earliest_filing_year + post_2011 (no interactions) or formula = n ~ earliest_filing_year * post_2011 (to include the interactions)