New answers tagged time-series
0
votes
What are good ways to statistically test for noise?
Much of this answer is copied from an answer I have given in this related question.
If you have data with seasonal effects in it, you would usually fit a model with seasonal terms and then extract ...
- 102k
1
vote
Measuring events in sales time series
What you normally do in such situations is to apply randomized A/B testing. I.e., you randomly separate your customers into two parts $g_A$ (the control group) and $g_B$ (the treatment group), and ...
- 8,694
1
vote
Is it possible for long-run variance to be negative or 0?
Remember that long-run variance is a limit, so it is a little bit different than what one regularly thinks of as variance. For simplicity lets assume $\mu=0$
$$\lim_{T\to\infty}\{\text{Var}[\sqrt{T}\...
- 1,041
0
votes
Accepted
Testing the impact of a events on time series
Sounds like Google's CausalImpact method might be useful (python implementation here). Briefly, you use pre-sale data to train a model that forecasts impressions, then forecast them for the sales ...
- 618
0
votes
Principle Component Analysis for Feature extraction from Voltage and Current Signals
The answer is yes. One can use sequence or timeseries generated by PCA for fault classification in transmission lines. Here is the story. I generated a fault sample in Simulink/MATLAB, saved data to ...
1
vote
Accepted
How to calculate variance of AR(1) process
The first step is to write the difference in terms of the original model:
$$ \Delta Z_t = Z_{t} - Z_{t-1} = (\alpha-1)Z_{t-1} + \nu_t $$
Since $Z_{t-1}$ and $\nu_t$ are uncorrelated by the white noise ...
- 680
1
vote
Dickey-Fuller test significant => series stationary?
Another aspect that I recently became aware of (see particularly this thread Is the Dickey Fuller test one-sided or two-sided?) is that the DF-test can be interpreted as one-sided in the sense that it ...
- 13.5k
0
votes
Accepted
Minimizing an objective function with input variables from a correlated error term
You can do linear regression for $y = X\beta + \epsilon$ using a covariance matrix for $\epsilon$ that belongs to an AR(2) model, or to any ARMA(p, q) model for that matter.
You can e.g. use the ...
- 8,694
1
vote
Resampling a timeseries
These returns are not iid, because:
they aren't identically distributed (have different variances, even if having the same absolute or scaled mean),
nor independent (the latter because their ...
- 714
0
votes
Daily Data Transfer Logs - Anomaly Detection
What you are proposing to create are called statistical control or Shewhart charts. There are many references, I would recommend Chapter 14 of Box's book "Statistics for Experimenters".
In ...
2
votes
Sum of $I(1)$ and WN
It seems to me that
$$
\begin{aligned}
Y(t)&=X(t)+w_t, ~~~w_t \sim N(0,\sigma^2_w) \\
& = X(t-1)+e_t+w_t, ~~~ e_t \sim N(0,\sigma^2_e),\\
& = (Y(t-1)-w_{t-1})+e_t+w_t
\end{aligned}
$$
is ...
- 2,124
0
votes
Do I need stationary time series data for Isolation Forest Model?
Tree based algorithms may struggle with non-stationary time series data. Especially when there is trend present, the future values of the series may have a very different distribution. For seasonality,...
- 53.2k
2
votes
Is the Dickey Fuller test one-sided or two-sided?
First, $\phi$ is the true coefficient of the process, not the test statistic.
The test statistic that is something else than $\phi$, namely the default t-statistic for the hypothesis that the ...
- 27.8k
0
votes
Can anyone point me towards tutorials describing how to use the Kalman filter for forecasting?
The best I found till now is this open source interactive book: Kalman and Bayesian Filters in Python
Hope this helps !
0
votes
Creating synthetic data for time series, Hidden Markov Model
If you know the parameters $(\pi, A, B)$ of the system you want to simulate, then it should be a pretty straightforward process.
Simply draw a uniform random number and compare it to the probabilities ...
- 41
1
vote
State space models in python statsmodels: including lag state innovation in observation equation
The problem here is that the $\theta$ and $\sigma_\varepsilon^2$ parameters are not separately identified. In particular, if you work through the Kalman filtering equations, it can be shown that the ...
- 1,288
0
votes
Interpret the Conditional Sigma (vs Realized Absolute Returns) of the DCC model
These seem to be the fitted conditional standard deviations (or perhaps variances) of four time series. On average, you would expect higher absolute returns when the distribution has a higher standard ...
- 57.8k
4
votes
Accepted
Simulating AR(2) Process With Initial Conditions
Programming the simulation from scratch requires just a few lines of $\textsf{R}$ code:
...
- 2,510
-1
votes
Accepted
Understanding the loss funcion in DeepAR
I have a better understanding of the training process now. I wrote it down and it is shown as images.
- 211
0
votes
Preventing information leakage when scaling a time series?
No... for training data for time-series data you cannot do whatever you like. The original reasoning is correct, you are introducing leakage into your data if you are not scaling with only past data. ...
0
votes
What do the "coefficients" in R's HoltWinters function represent?
Coefficients in Holtwinter's Method
In the HoltWinter's model with seasonal= 'additive', I am able to produce the model output for coefficients manually. It seems like we need to use a[N-p],b[N-p] ...
- 1
0
votes
Multiple Time Series Prediction (Python)
For Python - you can use https://pypi.org/project/keras-generators/ to generate multiple times-series input and target data for training.
It performs normalizing/scaling of data, train/test splits and ...
- 101
-1
votes
Is regression a good model for predicting disk usage growth?
If you want to use regression, I recommend using the scikit-learn library. It's quite straightforward:
https://scikit-learn.org/stable/modules/generated/sklearn.linear_model.LinearRegression.html
<...
- 269
2
votes
Accepted
Logit transformation of target values in regression
In what follows, I assume that the rate is bounded by 0 and 1.
This would map the rate to the log odds space, wherein the argument (rate / (1-rate)) would not be constrained to the unit interval. ...
- 27.9k
2
votes
Accepted
Deal with quarterly or monthly seasonality in forecasting a year ahead
In forecasting, one typically does not remove seasonality, but models it with a seasonal model. The tag wiki contains pointers to literature; I especially recommend the online textbooks by ...
- 101k
1
vote
Accepted
Computing reconciled prediction intervals when forecasting logged outcome variable using fable
The latest development version of fabletools allows you to probabilistically reconcile any forecast if you use sample/bootstrapped forecasts. Try updating the ...
0
votes
Does Box-Cox need to be Reversed Prior to Running Model Evaluation?
The process for ARIMA is to transform de data. Train the model, make prediction and reverse transform the data that has been predicted. If any other transformation has been made, like differentiation, ...
2
votes
Accepted
How do we make predictions for future data when you have lagged dependent features used in training?
Assuming your MAPE is on a test set and not based on the fitted values:
Yeah using features that are your lagged target variable is pretty common when doing time series forecasting with LightGBM ...
- 1,405
2
votes
Accepted
How do we compare MAPEs?
Certainly you can say that. You could also say that M2 is $1-\frac{\text{MAPE}_{M2}}{\text{MAPE}_{M1}}=30\%$ better. It depends on what the denominator of your percentage calculation of "better&...
- 101k
0
votes
Dickey-Fuller test for stationarity
It is because when $\phi > 1$ it's clearly non-stationary, as it has trend. In this case, the failure of stationarity can be detected by eye, so the ADF test is not necessary.
More importantly, the ...
- 257
2
votes
Accepted
How to correlate on sharp data changes
The simplest way would be to take the absolute value of the difference of each pressure value to its last value $\Delta p(t) = p(t)-p(t-1)$ and then calculate the correlation between this finite ...
- 713
0
votes
Accepted
Difference between seasonality-adjusted data and the trend component
An additive time series decomposition expresses a time series as
$$y_t = T_t + S_t + R_t$$
where $T_t$ is the trend component, $S_t$ is the seasonal component and $R_t$ is the remainder.
The ...
- 53.2k
2
votes
AR And MA Order
From your plots it appears that you might have a series with a periodic signal in it. The best way to check this is to look at the Fourier intensity of the time-series, which you can do using the <...
- 102k
0
votes
Rolling autocorrelation vs whole series autocorrelation
You should have a look at locally stationary processes. They are defined as typical time series models (e.g. ARMA) but where the parameters of model change smoothly over (unit interval) time.
In ...
- 2,779
0
votes
Does it make sense to increment by 1 the numerator and denominator in the MAPE to avoid division by 0?
Well, you can do this. I would very much recommend you take a look at What are the shortcomings of the Mean Absolute Percentage Error (MAPE)? The MAPE elicits forecasts that can be quite far away from ...
- 101k
0
votes
Accepted
Regression Analysis of Time as Independent Variable in Experiment
Running a simple linear regression model on the 300 pairs is a viable option. However, as you suspected, it would presume that there is no difference between the individuals. So the next possibility ...
- 8,694
0
votes
Accepted
Choosing the best Time Series Decompositions technique for International Trade
All of these are well-established methods. If one of them were clearly superior across a wide range of datasets, it would have superseded the others. Thus, which one performs best will depend on your ...
- 101k
1
vote
Goodness of fit for exponential model in lmfit
I only have one (depending on the perspective maybe 2) measurements per timepoint
Having only one measurement per timepoint brings you in a difficult situation. This makes it difficult to analyze ...
- 51.8k
1
vote
Goodness of fit for exponential model in lmfit
I'd leave $R^2$ out since your nonlinear models under comparison may have a different number of parameters. I would, instead, rely on one of these three approaches:
(a) use of information criteria ...
- 1,497
1
vote
Accepted
Does the Augmented Dickey-Fuller test only consider first-order effects?
It is related to the concept of stationarity of AR(p) processes, for which all roots of the characteristic polynomial
$$
\phi(z)=1-\phi_1z-\ldots-\phi_pz^p
$$
must be outside the unit circe, see e.g. ...
- 27.8k
0
votes
Aggregated time series or regression approach?
There's a problem with aggregating data (Approach 2) in this context: the survival estimate for an "average" set of covariate values isn't necessarily the average of survival estimates among ...
- 69.1k
1
vote
Accepted
Can I use interrupted time series analysis on irregular reported medical data?
Electronic healthcare records provide longitudinal data that don't have the regular observation times of panel or standard time-series data. Even if formal methods of interrupted (regularly spaced) ...
- 69.1k
1
vote
How to account for the pandemic in time-series forecasting?
If your time series exhibits 'anomalous behavior' in more recent observations, you necessarily have to remove or at least 'adjust' the data. Otherwise, your model performance will suffer for future ...
- 136
Top 50 recent answers are included