Questions tagged [moving-window]
A window is a fixed-length subset of consecutive observations of a time series. The window is moved along the time series at a constant rate. AKA "rolling window".
108 questions
22
votes
3
answers
8k
views
I'm getting "jumpy" loadings in rollapply PCA in R. Can I fix it?
I have 10 years of daily returns data for 28 different currencies. I wish to extract the first principal component, but rather than operate PCA on the whole 10 years, I want to rollapply a 2 year ...
- 641
11
votes
4
answers
7k
views
Tuning an exponential moving average to a moving window mean?
The alpha parameter of an exponential moving average defines the smoothing that the average applies to a time series. In a similar way, the window size of a moving window mean also defines the ...
- 5,541
9
votes
3
answers
19k
views
How to determine moving window size?
I am using moving window technique for data analysis...
For example I compute the mean, the standard deviation and etc. for a given window.
And I wonder if there's any good criterion to determine ...
- 1,251
9
votes
1
answer
2k
views
What is the autocorrelation function of a time series arising from computing a moving standard deviation?
Say I have a time series of observations and I compute a measure of the variance of that time series as the standard deviation (SD) in a rolling window of width $w$ and that window is moved in single ...
- 50.4k
9
votes
1
answer
15k
views
How to decide moving window size for time series prediction?
I have a model to predict +1 day ahead of this time series.
Looking at the chart you can notice some seasonality every 5 days. I suspect using a moving window as training set could help me making a ...
- 1,250
6
votes
1
answer
1k
views
Benchmarking time series forecasting model
Problem: I'm building a time series forecasting model for daily data wherein, the aim is to forecast for the next one week. So, to validate the model, I'm using a moving window based validation ...
- 253
6
votes
2
answers
4k
views
How can I estimate the sliding window standard deviation of a stream?
I am processing a stream of database records. At current levels, about 250 million records are added per week, but this will increase. I wish to compute the 90-day sliding window standard deviation of ...
- 226
6
votes
1
answer
357
views
DNA: The number of 'AAAAA'-s in a randomly generated DNA sequence that's 1000 base pairs long
Let's say I have a randomly generated sequence consisting of letters A, C, T and G that's 1000 letters long. The probability of each letter occurring is 25%. What is the probability that the sequence '...
5
votes
0
answers
690
views
Predict churn in a range of time after observation window is finished
I'm building a churn model. Each user's historic data (observation window) is a constant period, but each observation window contains different dates.
For example the next figure:
Let's say, that the ...
- 77
5
votes
0
answers
246
views
Can you use a moving average as an instrumental variable?
I have panel data and am interested in changes in total expenditures. I would like to consider an instrumental variable approach to deal with an endogenous regressor – the short run elasticity of ...
- 51
4
votes
1
answer
4k
views
Moving window PCA vs local PCA vs kernel PCA vs rolling PCA
All all these terms mean the same thing?
Are there other terms for MWPCA?
Are there any decent online references to theory and applications?
Which term is most popular (if they are equivalent)?
I'm ...
- 201
4
votes
1
answer
2k
views
Are rolling forecasts more accurate that full-sample forecasts?
I compared the auto.arima forecast checkts below to the rolling forecast fc and noticed ...
- 249
4
votes
2
answers
7k
views
Selecting ARIMA Order using Rolling Forecast
I'm wondering if a rolling forecast technique like the ones mentioned in Rob Hyndman's blogs, and the example below, could be used to select the order for an ARIMA model?
In the examples I've looked ...
- 125
4
votes
0
answers
179
views
improve performance of finding rolling window maximum likelihood
There is data indexed by time:
$$
D_1, D_2, D_3, ..., D_T
$$
I have a model that I assume the parameter $\theta_t$ changes with time $t$.
As a result, I adapt a rolling window strategy:
$$
\theta_{...
- 431
3
votes
1
answer
151
views
In sliding window regression, what is the best way to select my training window and test set size?
I am trying to forecast an index option's implied volatility using a sliding window regression and I'm a little confused on how I can go about cross validating with respect to the training and test ...
3
votes
1
answer
3k
views
Is moving average(sliding window) a smoothing technique or forecasting technique?
The rolling average method is mostly used to produce a smoothed series by removing noise. For ex- 3 window moving average, in general practice, the output for the fourth period is the 3 window moving ...
- 80
3
votes
2
answers
3k
views
Root-mean-square error when having multiple prediction horizons
I have a basic question about the root-mean-square error (RMSE). I have a prediction using an ARIMA model. I predicted a time series and use a rolling-horizon approach with overlapping or non-...
- 392
3
votes
1
answer
1k
views
Moving Average window
I am working with a multivariate time series problem which I try to predict 1 hour in the future. I am planning to use moving average of the features as separate new features. But I do not know if ...
- 679
3
votes
1
answer
978
views
Efficient online (rolling window) estimation of a GARCH model
I have a time series $x_t$ of length $n$. I would like to model it using rolling window approach with window length (width) $w$:
window $1$: $x_1,\dots,x_w$,
window $2$: $x_2,\dots,x_{w+1}$,
$\dots$,
...
- 69.5k
3
votes
1
answer
1k
views
Computation of multiple linear OLS regression with rolling window and/or update
How can I efficiently calculate an OLS fit for N multiple variables for a rolling window?
I've worked out how to do it for 1 and 2 variable linear fits, I'd like to extend to the general case of N ...
- 131
3
votes
0
answers
27
views
Determine a robust trend from noisy time series data, when start and end years have a material effect
I have about 20 years of data, each year has a number of observations. If I put a linear trend through the data, I get a trend, and this trend differs based on the the choice of start and end year, ...
- 131
3
votes
0
answers
159
views
Fixed vs rolling forecasts: Empirical evidence
Many sources online recommend that we use a rolling window to make forecasts. As I am in a choice between using a fixed window and a rolling window for my data I am trying to find any empirical ...
- 63
3
votes
0
answers
300
views
What are the potential problems with a rolling regression?
I have two time series that co-move, say a currency and a commodity, and the currency is highly impacted by the commodity, but also other factors. I'd like to determine when the currency becomes too ...
- 31
3
votes
0
answers
443
views
Good book / resource for learning about moving window PCA?
I would like recommendations on textbooks (or online resources) covering MWPCA theoretically and with application examples (preferably in R, but also in Python, Mathematica, Matlab).
A brief intro:
...
3
votes
0
answers
2k
views
Difference between recursive and rolling window estimation
I am trying to check if my Auto Regressive Distributed Lag (ARDL) model provides stable estimates over time. I am not sure if I should be using a recursive or rolling window method. I know that the ...
- 133
3
votes
0
answers
382
views
Correcting for multiple testing on non-independent sliding windows
Question
Values taken from adjecent windows in a sliding windows are corrolated. If I calculate a p-value from each window, how can I correct for the fact I have tested many windows, given that the ...
- 311
2
votes
1
answer
4k
views
Rolling vs Recursive vs Fixed Window Regression
What precisely are the differences between rolling, recursive and fixed window regression?
As far as I understand,
recursive: we train on a period $y(0)$ to $y(n)$ then predict $\hat{y}(n+1)$. Then ...
- 243
2
votes
2
answers
262
views
Rolling sum of 2 sample KS test results
In order to compare two lists of samples from 'before-treatment' and 'after-treatment', I am doing a two sample KS test using the ks_2samp function from Python's scipy.stats package which gives me the ...
- 23
2
votes
1
answer
4k
views
Window models in stream data processing
Reading about data stream clustering I met the next terms:
landmark window model,
sliding window model,
damped window.
As to sliding window it's clear - oldest data escape the scope, the new data go ...
- 123
2
votes
2
answers
902
views
Optimal window size for contextual outlier detection
I am looking for methods to detect univariate contextual outliers in time series data. One example application is data from industrial plants in different (unknown) operation modes or slow trends or ...
- 153
2
votes
1
answer
6k
views
ARIMA + Rolling Window
I'm currently working on building an ARIMA+GARCH model using R. My dataset consists of the logarithmic returns of the Dow Jones index for a period of 11 years 2005-2016, however, it's worth noting ...
- 23
2
votes
1
answer
2k
views
Detrending using moving average
In Terrence C. Mills book Im currently reading The Foundation of Modern Time Series Analysis on while discussing Person's detrending methods, he mentions the following result.
... Concider the ...
- 892
2
votes
1
answer
435
views
Lag selection and model instability for ARIMA-GARCH in rolling windows for forecasting
I'm to produce rolling forecasts with an ARIMA-GARCH model using a moving window size of 1000. Given that structural changes in the series might take place at some point in the forecast horizon, is ...
- 33
2
votes
1
answer
643
views
Impact of window size on estimated volatility using SMA or EWMA
When calculating volatility (either using an SMA or EWMA approach), what impact does the window size have on the volatility estimate?
- 163
2
votes
1
answer
228
views
Expected value of rolling variance/standard deviation of an AR(1) process
Consider a random variable following an AR(1) model:
$$x_t = \mu+\rho x_{t-1} + \epsilon_t$$
Assumme that $\epsilon_t$ follows $N(0,\sigma^2_\epsilon)$. Now consider the rolling variance and/or ...
- 139
2
votes
1
answer
230
views
What is the relationship between average of a rolling intercept and the intercept from a regression over the entire period?
If i calculate rolling (e.g. 3 periods back) intercepts for a time series using OLS, is the average of these rolling intercepts then in some way related to the intercept from an OLS of the entire ...
- 73
2
votes
1
answer
68
views
Ranking with multiple weights/ features
We have entities where every entity has start ($s_i$) and end ($e_i$) times and count $c_i$.
An entity is important if its interval ($e_i - s_i$) is large and if its $c_i$ is large.
Here's what I ...
- 175
2
votes
1
answer
837
views
How to compute the change of Ridge regression solution when one row of data changes?
I understand that $\boldsymbol{\beta} = (X^TX + \lambda I)^{-1}X^T\mathbf{
Y}$ is the closed form solution of Ridge regression.
So sometimes, when I run a rolling window, meaning everytime I run the ...
- 41
2
votes
1
answer
282
views
Markov Switching GARCH - Expanding or Rolling window forecasting?
When modelling volatility do people tend to use expanding or sliding windows to predict the performance of MS GARCH models?
- 265
2
votes
2
answers
1k
views
Supervised learning: setting labels on sliding windows of sensor data
Suppose that I have a set of accelerometer data collected with one sensor and one label for each measured data point. These labels describe different states of my system e.g., $state_A, state_B, ...
user185498
2
votes
1
answer
439
views
Time Series: relating duration of one time series to events
I have a stats question, and not enough experience to even begin to know how to find the answer. It's a time series question relating two time series:
A = anomolous conditions (-,0,+) --- continuous ...
- 31
2
votes
1
answer
360
views
Discovering peaks/patterns in time-series and clustering them
I have a dataset which contains minute level sensor measurements. Sample is shown here:
To me useful information are these peaks in time series, mostly their peak and duration. My idea is to take out ...
- 21
2
votes
0
answers
87
views
How does choosing an even window size actually add a cyclical component to the model?
I am new to Time Series Analysis. Say, we have a time series $(y_{t})_{t}$ that we want to filter with a moving average filter. I have been told that we should choose the window size $L$ of the filter ...
- 139
2
votes
2
answers
658
views
Rolling autocorrelation vs whole series autocorrelation
Suppose we have some financial time series. When we calculate the standard ACF, $\mu$ is considered as the average of all series' values. However, if we have a volatile series, the average can be ...
- 21
2
votes
2
answers
1k
views
Rolling Window Forecasting with ARIMAX while supplying actual values
I am comparing different exogenous variables in how good they support the forecast of the monthly seasonal adjusted unemployment rate. All my data is monthly (2006-01-01 until 2018-09-01) and ...
- 21
2
votes
2
answers
335
views
DNN methodology and feature concatenation
I'm using someone else's job and I have a question that I cannot solve.
This work uses a DNN to match an electrical resistance to a bend angle. This is not very important, just for the context.
So, ...
- 121
2
votes
1
answer
41
views
Using a recoded window 10 days back in a GAM-model, is that a double whammy or is it OK to do?
I'm formulating a prediction in a logistic reression type in a GAM model.
...
2
votes
0
answers
55
views
Picking the right prediction model
I am having a hard time trying to make/pick a prediction model in R.
The data: I have information on 40 different players, with all their recorded performances(training loads) over the last season. ...
- 21
2
votes
0
answers
303
views
Use sliding window to find variance for seasonal time series in R
I would like to estimate the variance of a time series. Say, if the time series has a period of 24, and I want to estimate the variance using
$$ \sigma_t^2 = \frac{1}{2k+1} \sum^k_{-k} (y_{t+24k} - \...
- 559