Can AIC/BIC be calculated reliably for a segmented regression model?

If so, can the results be compared against AIC/BIC calculations for non-segmented regression models on the same data?

  • 1
    $\begingroup$ I think you can do that. As long as you (1) keep the dependent variable exactly the same and (2) are able to calculate the degrees of freedom in the AIC/BIC penalties, you should be fine. $\endgroup$ – Richard Hardy Mar 31 '18 at 18:50
  • 1
    $\begingroup$ Are you including the breakpoints in the estimation procedure or are they externally specified? $\endgroup$ – jbowman Mar 31 '18 at 19:12

In principle yes. If you know the segments (i.e., if they are given exogenously), then the segmented regression model just corresponds to a certain kind of interaction model and information criteria like AIC and BIC can be applied in the usual way. Examples for time series data could include regressions where different coefficients are fitted for different known phases/regimes (e.g., governments, laws, etc.). For a regression with $k$ parameters and $m + 1$ regimes, the number of degrees of freedom is simply $\mathit{df} = k \cdot (m + 1)$. As usual, the penalty in the AIC is then $2 \cdot \mathit{df}$ and in the BIC $\log(n) \cdot \mathit{df}$ where $n$ is the sample size. (Some papers/textbooks also divide this by $n$, depending on the formulation of AIC/BIC.)

If the segments are not known but the $m$ breakpoints between the $m+1$ segments have to be estimated from data, information criteria are also applied routinely but there is more variety how to penalize the degrees of freedom for this. Often the breakpoints are accounted for by 1 degree per breakpoint, i.e., $\mathit{df} = k \cdot (m + 1) + m$.

This is used, for example, by Bai & Perron (2003, Journal of Applied Econometrics, doi:10.1002/jae.659) and they find that the AIC performs poorly and the BIC somewhat better. A modified BIC proposed by Liu, Wu, Zidek (1997, Statistica Sinica, http://www3.stat.sinica.edu.tw/statistica/j7n2/j7n213/j7n213.htm) performs even better, though. Instead of $\log(n)$ this uses $\alpha \cdot \log(n)^{2 + \delta}$ with $\alpha = 0.299$ and $\delta = 0.1$. Thus, there is a higher penalty than for the standard BIC if $n > 20$. (Bai and Perron actually conclude that sequential testing performs best for selecting the number of breakpoints, better than any information criterion they considered.)

A more recent study by Hall, Osborn, Sakkas (2013, The Manchester School, doi:10.1111/manc.12017) finds that the increased penalty in the Liu, Wu, Zidek criterion sometimes has problems finding true breaks and, instead, they recommend to use the standard BIC but with a different computation of the degrees of freedom. Instead of $\mathit{df} = k \cdot (m + 1) + m$ they recommend $\mathit{df} = k \cdot (m + 1) + 3 \cdot m$, i.e., accounting for each estimated breakpoint by $3$ degrees of freedom instead of just $1$.

My personal take on this is that there is some consensus that a large enough penalty term is needed in these situations. However, there is still some debate whether the usual BIC suffices or whether instead the penalty factor should be increased (as proposed by Liu et al.) or the degrees of freedom increased (as proposed by Hall et al.).


Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.