Quandt (1958) Endogenous vs Exogenous Breakpoint I would like to explain why an endogenous breakpoint is a problem when testing for a difference in the mean between two subperiods in a time-series. Here are my questions.


*

*The breakpoint cannot ever be chosen by the researcher, right? Even if the researcher has a "strong feeling" they "know" the "right" breakpoint, this is still endogenous, right? What's the difference between endogenous and exogenous, then?

*Quandt (1958, p. 877) describes the problem of an endogenous breakpoint as, "...the determination of t from the data reduces the degrees of freedom...of the variance ratio." In other words, the critical value of the "usual" test statistic is too small. Is there an easy way to describe how or why this is so?

*I would also like to re-frame the "endogenous breakpoint" problem as a "selection bias" problem in the Rubin Causal Model sense. In other words, there is selection bias unless the breakpoint is exogenous, e.g., found by the Quandt/Andrews supF approach. Is there a way to explain how an endogenous breakpoint creates selection bias and how or why the Quandt/Andrews supF approach overcomes the selection bias problem?
 A: 1.
The researcher may have an exogenous event in mind, and wants to test if there was a breakpoint corresponding to this event. Assuming that the likelihood function satisfies some regularity conditions then the researcher can use the likelihood ratio test to compare the hypotheses $H_0$: There is no breakpoint at time point $\tau$ versus $H_1$: There is a breakpoint at time $\tau$. (Wilks, 1938) is a common citation for this kind of analysis (although Wilks doesn't specifically refer to breakpoints).
2.
The "usual" test statistic is too small here because we have introduced a new parameter, the breakpoint $\tau$, which we choose in order to maximize the difference between the two resulting segments: equivalent to choosing $\tau$ to maximize the resulting test statistic. If $\tau$ was known ahead of the analysis, the usual test statistic would be fine, but the introduction of this new parameter inflates the test statistic.
The unknown breakpoint $\tau$ only appears under the alternative hypothesis so a likelihood ratio test with $\tau$ estimated from the data would not possess an asymptotic $\chi^2$ distribution (Davies, 1977; Andrews, 1993). Andrews (1993) calculates the asymptotic distribution of the maximum likelihood ratio when $\tau$ is estimated from the data, so a p-value for $H_0$ vs $H_1$ can be obtained.
The plot below shows the distribution of the absolute t-statistics for data generated from $100$ $N(0, 1)$ random variables (i.e. with no true breakpoint). If we guess that the exogenous breakpoint is in the middle of the time series (as discussed in Quandt (1958, p.877)) then we get a very different distribution of t-statistics than if we choose the breakpoint to maximize the likelihood. I'll put the code I used to generate this at the bottom of my answer.

3.
I can't see how the breakpoint estimation problem could be framed as a selection bias problem but I only have a little experience with selection bias problems and no experience with Rubin Causal Models.

Andrews, D. W. K. (1993). Tests for parameter instability and structural change with unknown change point. Econometrica 61(4), 821–56.
Davies, R. B. (1977). Hypothesis testing when a nuisance parameter is present only under the alternative. Biometrika 64(2), 247–254.
Wilks, S. S. (1938). The large-sample distribution of the likelihood ratio for testing composite hypotheses. The Annals of Mathematical Statistics 9(1), 60–62.

SegmentRss = function(x) {
    residuals = x - mean(x)
    sum(residuals^2)
}

FindBestBreakpoint = function(x) {
    minRss = SegmentRss(x)
    minTau = 0
    for (tau in 1:(n - 1)) {
        rss = SegmentRss(x[1:tau]) + SegmentRss(x[(tau + 1):n])
        if (rss < minRss) {
            minRss = rss
            minTau = tau
        }
    }
    if (minTau == 0) {
        print("Error: breakpoint 0 chosen")
    }
    else {
        minTau
    }
}

set.seed(1)
N = 10000
absTStatsMidBreakpoint = numeric(N)
absTStatsChooseBreakpoint = numeric(N)
n = 100
for (i in 1:N) {
    # generate a time series of 100 N(0, 1) random variables
    x = rnorm(n)

    # if we fix a breakpoint tau at n/2 what |t-statistic|
    # do we get (assuming equal variances in both segments)?
    tau = n/2
    absTStatsMidBreakpoint[i] = abs(t.test(x[1:tau], x[(tau + 1):n],
                                    var.equal = TRUE)$statistic)

    # if we choose the breakpoint tau to maximise the likelihood
    # (i.e. by minimizing the residual sum of squares), what 
    # |t-statistic| do we get (assuming equal variances in both segments)?
    minTau = FindBestBreakpoint(x)
    absTStatsChooseBreakpoint[i] = abs(t.test(x[1:minTau], x[(minTau + 1):n],
                                       var.equal = TRUE)$statistic)
}

library(ggplot2)
df = data.frame(BreakpointType = rep(c("Exogenous", "Endogenous"), each = N),
                TStatistic = c(absTStatsMidBreakpoint, absTStatsChooseBreakpoint))
ggplot(df, aes(x=TStatistic)) + geom_histogram(aes(y=..density..),
       binwidth=.1, colour="black", fill="white") +
geom_density(alpha=.2, fill="#FF6666") + 
facet_grid(BreakpointType ~ .)

A: *

*The difference between an "endogenous" breakpoint and an "exogenous" breakpoint is that the former is inferred from the data being used in the estimation, but the latter is not.  Consider the following chart fragment, copied from [Macrotrends]:1

We can deduce that there was a breakpoint in this series around 1973 by looking at the data, and if we look at the underlying data we can pinpoint it quite well.  It's pretty obvious, but it's still an endogenous breakpoint, and a model fit on the underlying data should reflect the fact that, in addition to the parameters being fit (on whatever model is being used), we are also estimating the precise location of the breakpoint from the same data used to estimate the parameters.  
However, if we happen to know that on October 17, 1973, Arab oil producers cut production by 5% and instituted an oil embargo against the U.S. and other countries, we no longer need to deduce the existence and location of the breakpoint from the data underlying this plot, which is of West Texas Intermediate crude oil prices.  In this case the breakpoint is exogenous - its presence and location is determined from data not used in the model.  
In the case of something this obvious, we might well assume that there must have been an event and not bother with degrees of freedom adjustment, but that's another topic.


*Alex's answer is correct and hard to improve upon.  I'll expand it to the related issue of t-statistics on the non-breakpoint parameters by constructing an example.  The actual data generating process is $y_t = e_t$, $e_t$ i.i.d. Normal(0,1), but we are going to estimate the model $y_t = \beta x_t + e_t$ instead, with and without a breakpoint also estimated from the data.  The breakpoint is a constant shift in $y$, so the model with breakpoint $\tau$ is $y_t = \alpha 1(t>\tau) + \beta x_t + e_t$.  The trouble comes about because $\tau$ is estimated from the data.  Our null hypothesis is that $\beta = 0$.  We generate the $x_t$ from a Normal(0,1) distribution, uncorrelated with the $e_t$ of course:
tstat <- rep(0,10000)
for (i in 1:length(tstat)) {
   y <- rnorm(50)
   x <- rnorm(50)
   tstat[i] <- summary(lm(y~x))$coefficients[2,3]
}
mean(abs(tstat)>1.96)
[1] 0.0539

for (i in 1:length(tstat)) {
   y <- rnorm(50)
   x <- rnorm(50)
   best_std_error <- 1e99
   for (bp in 5:45) {
      ones <- c(rep(0,bp), rep(1,50-bp))
      m1 <- lm(y~x+ones)
      std_error <- sd(m1$residuals)
          if (std_error < best_std_error) {
             best_std_error <- std_error
             tstat[i] <- summary(m1)$coefficients[2,3]
      }
   }
}
mean(abs(tstat)>1.96)
[1] 0.0702
> sqrt(0.95*0.05/10000)
[1] 0.002179449

Note that the initial t-test for $\beta = 0$ has about the right rejection rate; 0.0539 is 1.79 standard deviations above 0.05, but the rejection rate of 0.0702 when we are simultaneously using the data to choose and estimate the magnitude of the breakpoint is 9.27 standard deviations above 0.05, indicating that we will reject the null hypothesis of $\beta = 0$ too often - even though the breakpoint is on a different variable (the constant term, in this case.)  


*Selection bias.  One can make an analogy to selection bias, but it's not clear to me that it's a useful one.  The classic selection bias arises when some characteristic of the individual related to the outcome of a test is also related to whether the individual is placed in the control or test group.  In this case, we have two groups - the pre-breakpoint and the post-breakpoint group - and, when determining if and where a breakpoint occurs from the data, clearly each data point has some (small) influence on that decision.  Consider our example above; if $y_{38} = 2.1$, it's going to tend to cause the breakpoint estimate to be lower than $\tau=38$, and will also tend to cause the post-breakpoint estimate of the constant term to be $> 0$. Therefore each data point both has some influence on whether it's selected into the pre-breakpoint or post-breakpoint group and has some influence on what the estimated value of the relevant variables are in whichever group it's assigned to.


As Alex observes, the supF approach constructs a test for whether there is a breakpoint that corrects for the fact that we searched for the "best" breakpoint using the same data as we are using to calculate the likelihood ratio used in the test.  It doesn't completely eliminate the selection bias problem - after all, if you test at a 95% level of confidence, you're expecting to have a false positive about 5% of the time, and, even if the null hypothesis is false, the data itself may cause you to mis-estimate where the structural break is.  However, it eliminates that part of it caused by the fact that our search for the breakpoint results in biased test statistics relative to the classic situation where we are testing one, and only one, location for the breakpoint.
