# Statistical significance of outlier in time series

I have a time series for which I observe a new data point. I have reasons to suspect it to be an outlier (unsurprisingly, because of the COVID pandemic). Visually, this is confirmed. What I am interested in, is doing some hypothesis testing: I want to confirm, statistically, that the last year has been an unusual one.

How do I test & report this in a statistically sound way? Would I make a forecasting model and compare my new observation with the forecasting confidence interval? Or how would you do this? An important thing to note, here, is that this time-series is non-stationary, so a population mean is not well-defined.

Some thoughts on "outliers."

A similar situation can occur with data that are not a time series. I will illustrate a few of the issues about "outliers" using a normal sample.

Suppose you have $$n = 100$$ observations from a normal distribution with unknown mean and variance. Take the fictitious data in the vector x below as an example.

summary(x)
Min. 1st Qu.  Median    Mean 3rd Qu.    Max.
38.97   47.82   50.45   50.18   52.89   62.13
length(x);  sd(x)
[1] 100
[1] 4.492146

Then a 95% t confidence interval for the population mean $$\mu$$ from which data in x were randomly sampled would be $$(49.29,\, 51.07).$$

CI = mean(x) + qt(c(.025,.975), 99) * sd(x)/10;  CI
[1] 49.28708 51.06976

This is the same 95% CI that is given along with t-test:

t.test(x)\$conf.int
[1] 49.28708 51.06976
attr(,"conf.level")
[1] 0.95

Now you get a new observation $$X_{101} = 63.22,$$ presumably from the same population from which you sampled to get 'x`. You notice it is not contained in the CI above. But that CI is supposed to contain the population mean $$\mu,$$ not necessarily new observation $$X_{101}.$$

The appropriate kind of interval for that would be a 95% prediction interval based on the original data x, of the form $$\bar X \pm 1.984\,S\sqrt{1 + \frac 1n}$$ or $$(40.90,\,59.46),$$ as follows. So we might be suspicious that $$X_{101} = 63.22$$ is an outlier because it is not in the prediction interval. And some people might call it an outlier.

mean(x) + qt(c(.025,.975),99)*sd(x)*sqrt(1+1/n)
[1] 40.90107 59.45578

It is not good statistical practice to get rid of 'outliers' unless you know of a good reason why they may actually be errors--due to equipment failures, data entry errors, truly impossible values (like a human with age 273 years).

It is not unusual for a normal sample of size $$n=100$$ to show boxplot outliers. In fact, the minimum value $$38.97$$ and the maximum value $$62.13$$ in the data summary above for x are both boxplot outliers.

boxplot(x, col="skyblue2", horizontal=T)

It is worth considering whether $$X_{101} = 63.22$$ should be considered an outlier. If it was sampled during the COVID-19 pandemic, then maybe it should be considered an outlier. But $$63.22$$ isn't much larger than $$62.13.$$

Below are boxplots of twenty samples of size $$n = 100$$ from the same population sampled to get x. Seven of the twenty samples show at least one boxplot outlier. The highest outlier in any of the samples was at $$66.40.$$ (In all, about 6% of the 2000 observations in the plot below fell outside the boundaries of my prediction interval, horizontal red lines in the figure.)

I hope you can see how difficult it is to know when to declare an observation as an "outlier," and what to do with an observation you think is truly extraordinary. I think it is appropriate for you to make note of the suspected outlier in your time series and to speculate that its unusual size might be due to the pandemic. But I'm not sure I would delete it from the dataset. (That might depend on the next step in my analysis. But one should always report deletions of outliers.)

Note: Here is R code used to sample the fictitious data x used in the sample above.

set.seed(528)
x = rnorm(100, 50, 5)
• Thanks Bruce for the very detailed answer! I think I phrased my question a bit poorly, though (I will edit) for two reasons: 1) a relevant feature of the time series is that it's non-stationary, so there is no well-defined population mean, and 2) I am, in this case, not looking for outlier rejection but rather hypothesis testing: I want to say with some confidence that the COVID pandemic had an impact on the metric I'm interested in. May 28, 2021 at 6:06
• Things to ponder: You can probably make a prediction interval for a value in a time series--allowing for cycle, trend. // Will inclusion of the 'outlier' make a difference btw rejecting or not in your test? // Any way to match the 'outlier' with previous similar values (time of week/month/year, similar item/price/distributor, etc.) // Just noticing it's 'different' and blaming yet something else on the pandemic sees a bit thin. What is your evidence for 'some confidence'? May 28, 2021 at 6:22