What are good ways to statistically test for noise?

Question

I have read that highly autocorrelated data (such as stock markets) can be problematic when creating forecasting models essentially because we may infer that there is some signal or underlying model when in fact we are making sense of noise and the system is just highly autocorrelated.

I am looking to model sunlight irradiance which is highly seasonal on a yearly and daily level. Is there a good test to show to what extent noise is inherent within the system so that I can understand what the feasible limit for forecasting would be? It may very well be a silly question.

Answer 1

This is one of the points of time series analysis. The third edition of "Forecasting: Principles and Practice" by Hyndman and Athanasopoulos, freely available on-line, is a superb introduction to practical analysis of time series. Chapters 5 and 10 are perhaps most directly related to your question, showing how to generate plots showing prediction intervals into the future based on your model of the time series. This page, a bit old now, has many other suggestions for study.

Answer 2

Much of this answer is copied from an answer I have given in this related question.

If you have data with seasonal effects in it, you would usually fit a model with seasonal terms and then extract residuals from the model as your estimates of the error terms (which the model assumes to be noise). This pushes things back a step, but obviously you still need to test to see if there are any further seasonal effects in the residuals.

One useful formal test for this purpose is the "permutation spectrum test" which tests the maximum signal intensity against its null distribution under the assumption of exchangeability of the values in the signal (see O'Neill 2020). This particular test does not make any assumption about the marginal distribution of the data, so it is not restricted to testing Gaussian time-series. The null hypothesis for the test is that the time-series values are exchangeable and the alternative hypothesis is that there is at least one periodic signal in the time-series. (You need to remove trends before applying this test, since an linear or polynomial trend in the data may show up in frequency space as low-frequency periodic signal.)

---

Implementation in R: You can use the ts.extend package in R to produce and plot the signal intensity for a time-series or conduct the permutation-spectrum-test. To show you an example of this, let's first produce a time-series with a periodic component.

#Generate periodic part and random part of time-series
set.seed(1)
n <- 1000
A <- rep(1:20, 50)
E <- rgamma(n, shape = 2, scale = 30)

#Generate time-series with periodic part
a <- 1
X <- a*A + E

It is simple to produce and plot the intensity of the series in the frequency domain to see if there are any "spikes" giving evidence of a periodic component.

#Show intensity of time-series
library(ts.extend)
INTENSITY <- intensity(X, scaled = TRUE)
plot(INTENSITY)

We can see that there are some spikes at particular frequencies in the Fourier domain, but are they big enough to falsify the assumption that this is exchangeable noise? To test this we implement the permutation-spectrum test and produce an appropriate plot. In the present case the test correctly identifies strong evidence that there is at least one signal in the data. (The p-value for the test is $p=0.001385$.)

#Implement the permutation-spectrum test
TEST <- spectrum.test(X)
plot(TEST)
TEST

        Permutation-Spectrum Test

data:  real time-series vector X with 1000 values
maximum scaled intensity = 3.5762, p-value = 0.001385
alternative hypothesis: distribution of time-series vector is not exchangeable 
(at least one periodic signal is present)