New answers tagged time-series
0
There is little else you can do but do a long-range forecast based on the data you have. If your last historical data is from four months ago and you need an hourly forecast for tomorrow, you will need to calculate a four-months-plus-one-day hourly forecast. Yes, this will probably be very inaccurate.
Don't invest your time and efforts in looking for fancy ...
1
This is an old question, but has no accepted answer, so let me offer my own.
Here is some data that, while not being exactly like yours, is close enough for our purposes.
Because the data are non-linear, I think a GAM might work well here. I'll use the mgcv library to first fit a simple gam which uses a smooth for time and an additive effect for age group ...
1
It is repeatedly training a new ARIMA model on each subset of data.
0
Your series visually suggests a number of level shifts suggesting three local means. Since the mean is changing ...the series is de facto non-stationary . The software you are using probably "GETS CONFUSED" when there are deterministic change points in the mean as the only remedy it has is to incorrectly (in this case ) suggest differencing when other ...
0
As a special case, you can try using fbprophet to predict the sales including the variable as a regressor in the model.
1. It captures the trend.
2. Captures the seasonality.
3. You can use add_regressor method to accomodate variable c in your case as a special event.However, not sure how other two variables will fit in the model.You can explore more or ...
0
I will assume you have something that looks like xy coordinates per second. You can essentially count up how many times an xy pair appears in your data and then map a color intensity to a count value.
1
(I'm not a Bayesian statistics expert, so take this response with a grain of salt)
I know of two ways to use MCMC methods for time series forecasting:
Use MCMC to estimate the future forecast intervals or the future forecast distributions: in this approach, you use some other method (not MCMC) to generate the point forecast. Then you use MCMC methods to ...
0
No you should not include seasonality into your model, because then you are not asking how much variance of the expensive sensor can you explain using the cheap sensor, but you are asking how much variance of the expensive sensor can you explain using the cheap-sensor and seasonality data, which is obviously not what you want.
If you only care about R2 or ...
3
For time dependent regressors, it is pretty straightforward. Many classes of time series models can handle them, including from the ARIMA family (ex: ARIMAX and regression with ARIMA errors), BSTS, Facebook Prophet, and others.
The tricky part is time independent regressors: Most people don't realize that time independent regressors are of no use ...
1
Showing it is a Markov process means showing that
$$P(X_t = x_t | X_1 = x_1,\dots,X_{t-1} = x_{t-1}) = P(X_t=x_t | X_{t-1} = x_{t-1})$$
In your case, you have
\begin{align*}
P(X_t = x_t | X_1 = x_1,\dots,X_{t-1} = x_{t-1}) & = P(X_{t-1} + w_t = x_t|X_1 = x_1,\dots,X_{t-1} = x_{t-1})\\
& = P(x_{t-1} + w_t = x_t|X_1 = x_1,\dots,X_{t-1} = x_{t-1})\\
&...
3
Let $\{X_t\}_{t\geq 1}$ be a sequence of independent random variables such that $\Pr\{X_t=1\}=\Pr\{X_t=-1\}=1/2$. Define $\mathscr{F_t}=\sigma(X_1,\dots,X_t)$ and $M_t=X_1+\dots+X_t$. We have (equalities between conditional expectations holding almost surely)
$$
\mathbb{E}[M_{t+1}\mid\mathscr{F_t}] = \mathbb{E}[X_{t+1}+M_t\mid\mathscr{F_t}] = \mathbb{E}[X_{...
6
\begin{align}
E[X_{t+1} \mid X_1, \ldots, X_t]
&= E[X_t + a_{t+1} \mid X_1, \ldots, X_t]
\\
&= X_t + E[a_{t+1} \mid X_1, \ldots, X_t]
\\
&= X_t
\end{align}
0
Does it make sense to include the team ID as a feature in the model?
In most case yes. To your point, about a team's performance being correlated with rival teams (or with the performance of a team last year from which a star player had been transferred into the new team), the only the model can pick it up is if it is able to keep track of which data comes ...
1
$$ Cov(Y_t,Y_{t-k}) = Cov \Big(t \theta_{0} + \sum_{i=1}^{t} e_i, (t-k) \theta_{0} + \sum_{i=1}^{t-k} e_i \Big) \\
= Cov\Big( t \theta_{0}, (t-k) \theta_0\Big) + Cov\Big(t \theta_{0}, \sum_{i=1}^{t-k} e_i \Big) + Cov\Big(\sum_{i=1}^{t} e_i, (t-k) \theta_0\Big) + Cov\Big(\sum_{i=1}^{t} e_i, \sum_{i=1}^{t-k} e_i\Big) \\ = 0 + 0+0+ Cov\Big(\sum_{i=1}^{t} e_i, \...
1
Hourly predictions MAY depend on anthropormorphic activity
These might include
what day of the week it is
what month your are in
what level changes have occurred
what trend changes have occurred
what days of the month exhibit statistically usual effect
what recent activity has been *arima structure"
what week of the month you are in
holiday effects ...
0
Another popular question of mine. Well here's to self-help, starting with a sketch illustrating the 3 paths in the $N = 4, k =2$ situation:
The number of ways to arrange the 2 test sets to occur in 4 time periods is ${4 \choose 2} = 6$, and $\frac{k}{N} = .5$ is the fraction of the combinations that will start with a test set. Since a "path" is a continuous ...
0
I haven't seen anybody mention the book by Gloria Gonzalez-Rivera "Forecasting for Economics and Business". I have found it to be the best kept secret in the time series space. It is a terrific book. It will give you more intuition than Diebold, more context than Enders, and will actually be readable unlike Hamilton. With much of the outstanding literature ...
0
Last year I started teaching introductory and semi-advanced time series course, so I embarked on journey of reading the (text-)books in the field to find suitable materials for students. Given that I did not find any post on CV, Quora or ResearchGate that would full satisfy me, I decided to share my conclusions here.
This text below lists several time ...
0
You can do this with mcp. First, let's get your data in an accessible format. The variable "days" is the number of days since the first record. I remove the NAs:
library(dplyr)
D = read.table("bVSsxVtR.txt") %>%
rename(row = V1, date = V2, y = V3) %>%
mutate(days = as.numeric(as.POSIXct(date) - as.POSIXct("2014-10-31")) / (24*3600)) %>%
...
1
I'm assuming that s(time) or something like it in the model? If so, you can get into situations where both the smooth of time and the CAR(1) process, which are mathematically similar, cannot both be uniquely identified; one of the two ends up winning out, but which one can depend on a number of things. As you've found out it can depend on the scale of the ...
answered Jan 17 at 17:04
Reinstate Monica - G. Simpson
29.6k5 gold badges82 silver badges125 bronze badges
0
Yes, as long as all diagnostic tests for the residuals hold.
1
I read a paper a while back that used an L2-norm to do sort of a "temporal smoothing" process across the predictions, which I thought was an interesting way to deal with time-varying components. I unfortunately can't find it but here's a similar paper I found
Learning Time-Varying Graphs using Temporally Smoothed L1-Regularized Logistic Regression
http://...
0
The Box-Jenkins (ARIMA) model identification procedure consists of the following three stages.
Identification consists of using the data and any other knowledge that will tentatively indicate whether the time series can he described with a moving average (MA) model, an autoregressive (AR) model, or a mixed autoregressive – moving average (ARMA) model.
...
0
Stack it, i.e. write as VAR(1) for the vector x(t) = [y(t),y(t-1)]', x(t) = [beta1,beta2;1,0] x(t-1) + [epsilon(t),0]'. Vectorize the resulting matrix equation and exploit vec(ABC)=(C' kron A) vec(B)
2
Use mcp if you (1) want to quantify uncertainty about the location of the change point, and (2) want to specify a more informed model structure, e.g., that the first segment is a plateau. I arranged the data so that it is a regular data frame. Then fit an AR(1) model with a plateau + joined slope:
model = list(
y ~ 1 + ar(1),
~ 0 + x
)
library(mcp)
fit ...
0
For a Bayesian approach, you can use mcp to fit a Poisson or Binomial model (because you have counts from fixed-interval periods) with autoregression applied to the residuals (in the log space). Then compare a two-segment model to a one-segment model using cross-validation.
Before we start, note that for this dataset, this model does not fit well and cross-...
0
A primary issue with your 468 daily values is the presence of a number of anomalies. I took your data into AUTOBOX ( which I have helped to develop) and it simultaneously idenifies both the arima component ( 1,0,0)(1,0,0)7 and a number of evidented anomalies.
presents the Actual/Fit and Forecast. The model is here and here with statistics here
The ...
1
most probably (0,1,0) as stock prices are usually random walk models.
0
How to use Dynamic Regression models in R to forecast future sales should help . Note that VAR models don't allow for contemporaneous structure as all variables enter with only lags. You might also look at https://autobox.com/pdfs/regvsbox-old.pdf to get some perspective on "Regression vs Box-Jenkins".
0
I think what you're looking for are ARIMAX models. It's basically arima with xreg= argument including the exogenous variables.
Here are some links to shed some light on your tasks:
https://robjhyndman.com/hyndsight/arimax/
https://stackoverflow.com/questions/15681529/forecasting-using-arimax-in-r
https://cran.r-project.org/web/packages/forecast/forecast....
0
If your data for training and test are not in the same format, then this could cause issues when the model attempts to interpret features for the test set.
As an example, suppose one of the features is a categorical variable in string format.
apple
banana
blueberries
cucumber
orange
Part of the data cleaning process appends numbers to these categories:
...
2
You seem to be confusing some things, so let me clarify. First of all, Bayesian models do not have any special connection to time-series. Same as for non-Bayesian models, using Bayesian framework you can define possibly infinite number of different statistical models.
Sure, you can first estimate the model for $n$ time-series points, and then update it for ...
0
Feature engineering may leak all kind of information, so if you do it first and then split, you can end up with a model that does good on test data, but not on real data. So I don't agree with the other answer, you should split first.
0
It depends if it's an actual cleaning (missing data, wrong data, exceptional outliers) or stylized facts cleaning (i.e. smoothing or filtering, seasonal adjustment). Though, the general rule is to clean first. Results are likely to be different, but it ultimately depends on how much you've clean.
0
If you are a complete beginner as you state, run the multiple linear regression with the happiness score (if it is indeed a continuous and not a categorical variable) for each country and look at the coefficients and whether or how they change.
This answers the question "how was country x1's happiness score in 2015, 2016 and 2017, controlled for GDP and ...
1
Unevenly spaced time series is a term that is used. While most statistics theory is about evenly spaced time series. In the comments there is also proposed point process, but that seems to be a special case where only the times itself are observed and of interest.
So if the observations are occurrence times of earthquakes in California, that would be a ...
1
There are two types of Intervention studies . The first one is called Intervention Analysis (de jure )..the second is called Intervention Detection (de facto). Simply search here for R and one or the other.
The ultimate approach is to use a SARMAX model https://autobox.com/pdfs/SARMAX.pdf to form a useful equation leading directly to tests of statistical ...
1
SOLVED: Issue was the dummy variable trap. Removing one of the seasonal binaries when feeding data fixed the issue
0
You can do this using the mcp package, provided you know the number of segments in advance:
model = list(
price ~ 1 + sigma(1), # Intercept and variance
~ 0 + sigma(1), # Change in variance, but not in mean
~ 0 + sigma(1) # same
)
library(mcp)
fit = mcp(model, df, par_x = "time")
You can infer the time at which these changes take place on ...
0
You can use mcp for this kind of models. Time series often exhibit autocorrelation, so let's just to an AR(1) model with three linear segments. First, let's specify the model:
model = list(
y ~ 1 + x + ar(1), # Linear segment. Initiate AR(1)
~ 1 + x, # linear
~ 1 + x
)
Now fit it:
library(mcp)
fit = mcp(model, df)
As an extra trick, if the slope ...
0
This inference problem has many names, including change points, switch points, break points, broken line regression, broken stick regression, bilinear regression, piecewise linear regression, local linear regression, segmented regression, and discontinuity models.
Here is an overview of change point packages with pros/cons and worked examples. If you know ...
0
Answering out of order:
2 - Each gate is NOT a DNN, it is an LSTM Cell, which is a single neuron, albeit one whose inner workings are more complex than the basic neurons used in DNN. There are two key concepts in LSTM:
Recurrence, where the output is fed back to the input. This is a property of RNN's in general, of which LSTM is a special case. DNN cannot ...
answered Jan 10 at 4:31
Skander H. - Reinstate Monica
7,6831 gold badge21 silver badges52 bronze badges
1
I'm not sure what you mean by "fixed accuracy" in this context. A fixed accuracy would imply that since the accuracy is always the same (and known after the first few forecasting) we can adjust for the accuracy rate to recover the original value of the time series and we end up with a perfect forecast instead.
I assume what you really want is to simulate ...
answered Jan 10 at 3:37
Skander H. - Reinstate Monica
7,6831 gold badge21 silver badges52 bronze badges
0
R's forecast package now has a function mstl() to handle multiple seasonal time series decomposition.
This page has got more details how to use it: https://pkg.robjhyndman.com/forecast/reference/mstl.html
0
AR(2) is causal if :
$$ \phi_1+\phi_2 < 1$$
and
$$ \phi_1 - \phi_2 < 1$$
and
$$ -1 < \phi_2 < 1$$
In this conditions $ \phi_2(B)=0 $ equation roots are outside of unit circle so it's causal.
1
Similarities between countries would be based upon examination of models thus build the models first.
Trend analysis is at best a vague concept . Trends can be deterministic or stochastic (as part of an arima model). Either trend detection needs to be concerned with level shifts ( which are not trends ) . see ML preprocess to achieve stationarity and ...
0
I have a pretty big time series - daily time series of pollen observation gathered for 30 years. I tried to put my data and also improve a bit my question but I can not see any of my changes. I do not know why.
0
The table contains 3 methods of analysis:
"Moving Average": Simply the mean of the closest 4 data values. Example: The first value is $290.75 = (207.5 + 265.5 + 401 + 289) / 4$. This is called a moving average with centered window of size four; as for any point in time it takes the closest four datapoints to the left and the right of it and then moves in ...
6
stats::arima() estimates $\sigma^2$ using the MLE of the innovations variance, while forecast::Arima() uses the unbiased estimate $\sum e_i^2/(n-k)$ where $n$ is the number of observations available and $k$ is the number of parameters estimated.
stats::arima() does not count $\sigma^2$ as a parameter in the computation of the AIC, whereas forecast::Arima() ...
2
Yes it makes sense, and all employees should be considered (up to the time they leave)
I think you are slightly confusing the time to failure analysis:
Probability of failing in each time period is independent of what happened before. However, a basic model with no time dependence fitted to eg up to 5 year employees will sooner or later churn, so your 20 ...
Top 50 recent answers are included
