New answers tagged time-series
0
votes
PACF for MA(1) process
From Time Series Analysis, 1990 by William W. S. Wei book
\begin{eqnarray*}
\phi_{kk} =
\frac{ \det
\begin{pmatrix}
1 & \rho_1 & \rho_2 & \cdots & \rho_{{k}-2} & \...
- 101
0
votes
finding sparse regions in time series data
Interesting question! and interesting data. You can focus on the waiting times between baptism, so take the successive differences. Then you could make a local estimation of the mean waiting times (...
- 64.7k
2
votes
In a time series $x_t, x_{t-1},...,$, why is $E[x_t|x_t, x_{t-1},...]= x_t$?
As the comment by J. Delaney says, the result would hold even without conditioning on $x_{t-1}, x_{t-2},\cdots$.
The statement in the OP's question might be the result of a bad cut-and-paste job from ...
- 41.7k
0
votes
How to select a time series model based on this ACF?
This doesn't look like the ACF of a (short) MA process as it goes to zero too slowly. The next step would be to look at the partial autocorrelation function (PACF). If the PACF goes to zero reasonably ...
- 46
0
votes
Accepted
Conflicting ACF/PACF after first-difference
There is nothing conflicting here. Differencing a unit root process removes the unit root. If the original process is AR(1) with a unit root, $y_t=y_{t-1}+\varepsilon_t$, the differenced process is ...
- 55.4k
0
votes
How to predict multiple future values in a linear model in R?
To predict $credit_{t+1}$, you need $year_{t+1}$ (which you have) and $student_{t+1}$ (which you don't have). So first you need to create a model (or a formula) to predict that.
Looking are your data, ...
0
votes
How to find an instrumental variable
Finding an instrument is literally "art".
In a sentence, we just consider a variable satisfyimg the two IV condition, and no theory is applied.
If you can convince people, then that is ...
- 501
0
votes
Tensor Classification Models
Basically, your data consists of multivariate timeseries.
Since you have a time-varying dimension, you are looking at a few models that can do that.
1D Convolutional Neural Networks: convolutions are ...
- 15.7k
1
vote
Tensor Classification Models
You can flatten the tensor and run the usual machine learning methods on the vector: random forest, kNN, SVM, logistic regression, multilayer perceptron neural networks, etc.
This has been done for $2$...
- 30.8k
2
votes
Accepted
Inclusion of year and seasons as variable for regression with non-stationary response?
Regarding OLS only makes sense if both the response and explanatory variables are non-stationary, actually, it is the opposite: non-stationary should be replaced by stationary. Though as Chris Haug ...
- 55.4k
0
votes
Applying Dynamic Time Warping (DTW) instead of Euclidean Distance for Clustering Synchronized Time series data
Yes - if the time series are the same length and aligned, DTW will give you the same result as the Euclidean distance (ED).
There are two possible issues I can think of with using DTW. Firstly, in ...
- 422
2
votes
Applications of Dynamic Time Warping (Time Series)
DTW is an algorithm for measuring the distance between two time series. It's an alternative to the Euclidean distance (which is the mean squared distance between the time series at each time step), ...
- 422
1
vote
Churn model- how to handle new users without enough historic data?
There is a technical difference between 0 and NA, and new users should have a NA rather than a 0 for their number of visits in the preceeding weeks. I think some implementations of trees are able to ...
- 25.8k
0
votes
Analyze a time series to predict a value
There are a few approaches you can use for this type of regression model using time series data. I'll mention a couple that are commonly used in earth/environmental science applications. The first is ...
- 422
7
votes
Can I take a random sample of my very large data set to overcome non-independence?
I will add a couple of points to Tim's answer, focussing on the original question, which was "My question is - is this valid? Am I missing anything here?". I think the approach can be valid, ...
- 351
0
votes
Stationary time series having unusual ACF and PACF plots
Does it affect the AR degree=2 and AM degree =1 in the ARIMA model?
If it's daily data, then you should look up for what those lags around 34 could mean (economically, financially, etc), which depends ...
0
votes
Analysis on repeated measurements with only one follow-up outcome
It's probably best not to think of the health status as an outcome, because you have no information about its value prior to the study. Since diet-related health status typically changes very slowly, ...
- 61.9k
1
vote
Can I use non-stationary variables in forecasting problem
One big problem you face is the way that Cox PH models incorporate the effects of time-varying covariates. Those models assume that the current hazard of an event is related to the current covariate ...
- 61.9k
1
vote
Tuning ARIMA/ETS for univariate time series
A model with a better fit =/= a model with better forecasts in the time series world. Most 'auto' tuning frameworks optimize based on an approximation of the forecast accuracy rather than in-sample ...
- 1,265
1
vote
Shall I use daily or monthly data for demand forecasting?
It all depends very much on what you plan on doing with your forecast. If you use it for production planning, and your production plans are always frozen in monthly buckets, then you need a forecast ...
- 96k
0
votes
Turning sleep schedule data into a statistical distribution
Initial thought: Asleep versus Awake at time $t$ is a binary outcome.
Let
$ x_{it} \mid \pi_t \sim \text{Bernoulli}(x_{it}\mid\pi_t)$, then have $\pi_t$ be a probit transformation of a Gaussian ...
- 171
1
vote
Getting different AIC / BIC values for AR(2) estimation via AutoReg(2) vs ARIMA(2,0,0) through python statsmodels
When AutoReg was first included in Statsmodels in e.g. v0.12, it used the AIC definition from Lutkepohl's book New Introduction to Time Series Analysis, which ...
- 1,203
0
votes
How to approximate MAE of monthly values from MAE of daily values?
I've assumed that when you say the errors are normally distributed you mean the daily values (and so the monthly values) are normally distributed. I've also assumed that by MAE you mean the mean ...
- 422
1
vote
Turning sleep schedule data into a statistical distribution
Strictly speaking (in my humble opinion), you cannot build up an empirical distribution from the data to then detect outliers in the data (sort of a chicken and the egg problem). You can evaluate the ...
- 51
0
votes
Looking for advice: Short-term forecasting using actual forecasts and real time data
The first step is to organise all of your historical data and clean it - you say you have inconsistent reporting due to network problems, this will lead to gaps in your historical dataset which will ...
1
vote
Generalized Linear Models vs Timseries models for forecasting
I myself was studying neural behavior a long time and I must say that GLMs did a really good job in predicting the complex neural behavior based on external factors but also the activity of other ...
- 230
2
votes
Why is this linear combination of random variables from a white noise stationary?
There are technical issues associated with the infinite sum. Identifying and resolving them requires some attention to the definitions, so let's begin there.
A white noise process $\epsilon_t$ (in ...
- 287k
1
vote
Forecasting based on few samples
Your data can be plotted as follows:
Note: Always plot your data! Especially if you want to forecast.
In covid models, a V-shape recovery has been quite frequent.
The blue line is your data. The red ...
0
votes
Regression analysis with time series data
Conventional wisdom suggests that you should use ARCH (GARCH) models rather than OLS or ARIMA models, for the modeling of returns (percent change in prices). The literature of those models is very ...
0
votes
Accepted
How to formally test which fixed effects to use with panel data?
In this case, you should test for heteroskedasticity and autocorrelation primarily. One key reason to use any fixed effect model is to solve for heterogeneity, which is not exactly as ...
0
votes
p/q/d from ACF and PACF
Given that there were no determistic brekpoints in model error variance over time AND no changes in model paramaters over time AND that there are no pulses, seasonal pulses, level shifts or ...
- 28k
0
votes
Interaction terms and causal interpretation
You should accept the answer from @dipetkov (+1), as the difficulties of causal interpretation are key.
In terms of statistical analysis per se, note that $T_{t}$ is necessarily correlated with $X^{1}...
- 61.9k
0
votes
Accepted
AIC/BIC of ARIMA and ARIMA-GARCH
It is highly implausible that the AICs of such relatively similar models differ as much. Most likely they are not directly comparable due to quirks of definitions of AIC (and the likelihood on which ...
- 55.4k
2
votes
Accepted
Proof that a necessary condition for characteristic roots to lie inside unit circle is $\sum\limits_{i=1}^{n} a_{1} < 1$
Let
$$P(z) = z^n - \sum_{i=0}^{n-1} a_i z^i.$$
Notice that
$$P(1) = 1^n - \sum_{i=0}^{n-1} a_i 1^i = 1 - \sum_{i=0}^{n-1} a_i.$$
If all the roots are inside the unit circle, there are no roots on the ...
- 287k
4
votes
Accepted
Interaction terms and causal interpretation
Your model is a regression with a (linear) time component and a time-treatment interaction. Otherwise you explain nothing about the problem or the data you are working with. In this very general ...
- 2,131
2
votes
Accepted
Model performance when ground truth is not available
I am not sure I understand your first technique, because, AFAIK, the reconstruction error of autoencoders is what is used as the score for anomaly detection in the first place, so your first technique ...
- 2,843
0
votes
Need help interpreting ACF and PACF plots for ARIMA
Your reasoning is right for exclusive AR or MA process. But in this case, it's seems to be an ARMA Process with $p$ and $q$ coefficients. In this case the order can be quite challenging, but always ...
3
votes
Accepted
Can I take a random sample of my very large data set to overcome non-independence?
If you downsample time-series data it would not remove the dependence, it would just dilute it. Say that your data follows the relationship
$$
y_{t+1} = f(y_{t}) + \varepsilon_{t+1}
$$
so the current ...
Tim♦
- 113k
2
votes
how to deal with data leakage in historical data
I just came up with idea that:
It’s not realistic to use future data to predict the past. That’s not a real use case.
There could be some temporal related changes for players, and sanity wise, using ...
1
vote
Regression of a time series difference
That's the main difference between math and statistics in time series. If you assume that the actual data generating process (DGP) of $y(t)$ is your equation, then your reasoning applies. But if it's ...
1
vote
Stationarity and ergodicity of a process conditional on a finite trajectory
Your answer here is mostly technically correct (though it is of course possible for some time-series that you might get observed values $y_1= \cdots = y_t = \mathbb{E}(X_k)$ in some other applications)...
- 94.2k
2
votes
Accepted
Stochastic modelling, distribution and ergodicity of a particular time series with a given finite history
Your question misunderstands how conditioning works in measure-theoretic probability
I think you are letting the complexity of the notation for a stochastic process get in the way of intuitive ...
- 94.2k
0
votes
Chow F statistics using strucchange for ARMA(1,1)
The Fstats() function relies on fitting linear regression models (via least squares), internally. Thus, fitting models with moving average terms is not ...
- 13.9k
0
votes
Neweywest test in R
The lag in NeweyWest() (and also in kernHAC()) is selected automatically by default. Note that both of these do not apply simple ...
- 13.9k
1
vote
Recalculate fitted values/Simulate of an Arima model with different xreg values
You can use simulate(arima_model, xreg = new_x) with a the parameter xreg to simulate a time series with a different regressor ...
- 46.2k
1
vote
time series forecasting model cannot beat baseline
You didn't give us many details about your data, but if you don't have many observations then the simple methods like predicting the previous value work remarkably well. What is the frequency of your ...
Tim♦
- 113k
0
votes
Accepted
1 Stationary time series, 1 non stationary: do I need to transform BOTH, OR can I use VAR with 1 transformed and 1 stationary variable?
No, you do not have to transform the series that is already stationary. You can use one series transformed and the other not transformed.
Yes, you can build a VAR model for them and make forecasts.
- 55.4k
0
votes
Which model should I use for Time series data?
Richard. I hope this helps.
If it's indeed a time series data, it's a single loan or an aggregated loan, isn't it? Those characteristics have to be attributed to that single/aggregated loan as well. ...
1
vote
Detecting outliers in a multiple time-series
I am not sure what the notion of "jumps" refers to and how irregular they can be. For what follows, I simply presume that each of your $m$ price time series $x_t^{(i)}, i=1,\ldots, m$ is ...
- 2,843
1
vote
How to use time-series observations on multi-class classification problem?
One of the challenges of time series classification is that most ML algorithms assume the feature variables are independent of each other. That's often not the case in a time series, as a value of the ...
- 422
Top 50 recent answers are included