How to test if the mean of a time series has significantly shifted at a certain point?

Question

sorry if this has been asked before but I have searched the internet a lot but have not been able to find a satisfactory answer.

I have a time series (approximately 25 points) and I have implemented Binary Segmentation to detect a changepoint in the time series. My question is, how do I see if this changepoint is statistically significant? I.e I want to see if there is a shift in mean around that certain point in the time series. Ideally, I want to summarise this using a p-value. The standard t-tests do not apply I assume because the samples belong to a time series and hence the samples have some correlation.

I would be extremely thankful for any help. Thank you

Answer 1

This is a well-studied problem. Given the vagaries of time series data, there is no one-size-fits-all technique. If the purpose is to test whether there is a changepoint at a KNOWN location/time, a common method is the Chow test. But if the location of the changepoint is unknown, there are some methods like the Quandt Likelihood Ratio test and the Pettitt test for that purpose. All these procedures are available in R.

Further if the purpose is to explicitly test whether there is a shift in the mean, some model-based approaches may be more flexible. There are numerous packages available in the statistical program R for that, such as the changepoint and bcp packages. If somebody cares a Bayesian method, one choice is the Rbeast package developed by my group, available in R, Python, and Matlab (To indulge myself in self-prompting: https://github.com/zhaokg/Rbeast). Here is a quick example in R

library(Rbeast)
data(Nile)                    # the annual streamflow of the River Nile
o1 = beast(Nile)              # A full Bayesian model
o2 = beast(Nile,method='bic') # based on the Bayesian information criterion
plot(o1)
plot(o2)

Answer 2

In the simpler case of independent data points, a simple two-sample t-test here would give too-low p-values because you choose the change point to create the pair of datasets with the largest possible t-statistic.

Suppose we generate a time series of 26 observations of $N(0, 1)$ observations, with no change points. If we split the data in two in the middle, the p-value will be $U(0, 1)$. But if we do binary segmentation then the p-value is likely to be lower, even though there are no change points in the data. I've simulated this happening 1,000 times in the code below

N = 1000
n = 26
set.seed(1)
# p_vals_half gives the p-values of t-tests from splitting the data into two equal parts
p_vals_half = numeric(N)
# p_vals_bin gives the p-values after choosing the split with binary segmentation and then computing the p-value for the resulting t-test
p_vals_bin = numeric(N)
for (i in 1:N) {
    data = rnorm(n)
    p_vals_half[i] = t.test(x = data[1:(n/2)], y = data[(n/2 + 1):n], alternative = "two.sided")$p.value
	    # choose the change point that minimises the difference between the two data sets (and so minimises the p-value of a t-statistic)
	    pvals_all = numeric(n - 3)
	    for (j in 2:(n - 2)) {
		    pvals_all[j - 1] = t.test(x = data[1:j], y = data[(j+1):n], alternative = "two.sided")$p.value
    }
    p_vals_bin[i] = min(pvals_all)
}

par(mfrow = c(1, 2))
hist(p_vals_half, xlab = "P-Value", main = "Histogram of p-values from splitting in half")
hist(p_vals_bin, xlab = "P-Value", main = "Histogram of p-values from binary segmentation")

As you can see from the output, the straightforward t-test applied to a dataset generates a lot of spuriously low p-values. So even in the independent data case, you would need to take the fact that you are choosing the change point into account when you calculate the p-value.

I'd need more information about a model for the correlation structure of the data that you have to give a specific answer, but this paper, Change point detection in autoregressive models with no moment assumptions by Akashi, Dette, Liu (2016), may be useful. Usefully, it cites a lot of papers on the topic of change detection in autoregressive models.