New answers tagged forecasting
1
If you have enough data (and since it is an hourly period you do) there should be no problem. You can try some kind of time series cross validation where each resample goes to a different period as long as each resample has sufficient sample sufficiency. This way you will be able to compare
0
You can see the explanation in this video: https://youtube.com/playlist?list=PL3N9eeOlCrP5cK0QRQxeJd6GrQvhAtpBK
Hope it helps.
Regards,
0
I have gone through this before as well so I'll share a few learnings we had upgrading from traditional stat time series to boosted trees:
Feeding lagged target features to xgboost/lightgbm typically didn't work and added a lot of complexity due to needing to predict multiple times. Calendar features such as week number/ month number could cause a lot of ...
0
I don't have a fully satisfactory answer, but it may be helpful. If you really want to understand the details, you may want to step through the source code of forecast.Arima() (note the capital A).
Your arima() call fits an ARIMA(0,1,0)(0,0,1)[12] model. The formula for this model is
$$ (1-B)y_t = (1-\Theta B^{12})\epsilon_t, $$
where $B$ indicates the ...
1
First off, How to know that your machine learning problem is hopeless? may be enlightening.
The drivers of TV viewership that you are modeling are not very strong.
Weather may have been a useful predictor forty years ago or more, when there were only two TV channels, so when people were driven inside by bad weather, they only had a choice between these two ...
0
Finally, I did not include a constant. Instead, I've calibrated the cuts (without changing the utility function) in order to reproduce the market shares using the synthetic population of today. Then, I applied the calibrated model on the synthetic population for the future.
0
Q1-Q2:
Depends on the goal. If you are going to use model to forecast N days ahead then you should test your model by comparing N days ahead forecast and real data from N days ahead. Here is a code psedo code for 1 day ahead testing for a AR model:
https://stats.stackexchange.com/a/503021/293623
You can modify it to N days ahead.
Q3:
It does not affect the ...
0
As commented by other users, I needed to adjust the function to be arimafit_f = auto.arima(ts(kt_train_female, frequency = 12))
0
or "all models are wrong, some models are useful." When they say extrapolation is dangerous they are talking about cross sectional regression not time series. There are dangers when you estimate outside the range of your data. But if you don't want to extrapolate you can't do time series at all so its a meaningless point (well for those who do ...
0
No, you don't.
And I am not even calculating the correct MASE from your data - because I can't.
The MASE is calculated by diving your forecast's MAE (which for your "Naive Forecast" column would be 30.17) by the MAE achieved by the random walk forecast in-sample. Note that the page you link refers to "the training MAE".
And since we don't ...
0
You almost certainly have multiple-seasonalities: there will be pattern within days, but weekdays will very likely also differ from weekends. So your frequencies will be both 34 (the length of intra-daily cycles) and 7*34=238 (the length of intra-weekly cycles). You may also have other periodicities, like monthly or yearly ones, but these will likely be the ...
1
You cannot say anything about overfitting from (log-)likelihood, AIC or BIC alone. You could say something by comparing the model's in-sample fit with its out-of-sample prediction accuracy. If the in-sample residuals are subtantially smaller than the out-of-sample forecasting errors, the model suffers from overfitting; otherwise, it does not. Time series ...
1
Your ACF is (more or less) exponentially decaying.
Your PACF has a clear peak at a lag of 1, and much less clear ones at lags 2 and 34.
I recommend Forecasting: Principles and Practice (2nd ed.) by Athanasopoulos & Hyndman. Based on the section on ACF/PACF reading (scroll down), it looks like your data may be best described by an ARIMA(1,1,0) process.
...
0
jbowman already gave some very useful pointers. To follow up on your question:
If you were to intuitively explain the numerator and denominator, how would you do it?
I would call them both "nonstandard mean squared percentage errors", each period's error being expressed as a percentage of the previous observation. The numerator does this for the ...
0
Your prediction look like made by a random walk with drift model, Indeed the graph sound like it. Your result are quite usual. You do not make serious mistake, is not possible to find a much better ARIMA specification.
However is crucial to understand the main points of the problem, read here: Forecasting Prices vs Returns by Deep Learning
0
Given the volatility in a stock market time series ARIMA, is not an appropriate technique. If you really want to model the volatility in stock market data try GARCH instead.
If you want to learn about Time series forecasting I suggest picking up the Box-Jenkins book on the subject.
1
Stock prices are the worst kind of a toy problem to train yourself on when studying time series. The simple reason: they're essentially unpredictable. At least the kind of the series you are using in the time frame that you chose. Just think logically: if the price series were predictable why wouldn't everyone predict them and make money? See what Fama got ...
4
Forecasters (those who do worry about statistical significance, which is still not all of us; compare Diebold's 2015 recollection of "bewilderment as to why anyone would care about the subject" in the referee report to their initial submission) will often happily summarize multiple steps ahead to obtain a mean error per series and method across ...
0
Best can be defined many ways. More accurate is the way I define it as a practitioner not a researcher. I just compare the predicted results to the actual results with a MAPE. But even for accuracy there are many alternatives.
There is of course no test of statistical significance this ways. Since the results are real I don't really think that matters. No ...
0
I suggest using auto.arima if you are new to ARIMA. I use it and I have studied ARIMA for a while. :) It is FORECAST package in R. It quickly chose the model, something the classical methods struggle with when you have mixed models (that is both AR and MA).I am not sure what you mean by a massive difference. A massive difference in what? It could be there ...
1
Consider $X_{t+2}$ term:
\begin{equation}
\begin{split}
X_{t+2} & = \sum_{i=1}^p \psi_iX_{t+2-i} \,+Z_{t+2} = \psi_1X_{t+1} + \sum_{i=2}^p\psi_iX_{t+2-i} \,+Z_{t+2} \\
& = \psi_1\left(\sum_{i=1}^p \psi_iX_{t+1-i} + Z_{t+1}\right) + \sum_{i=1}^{p-1}\psi_{i+1}X_{t+1-i} + Z_{t+2} \\
& = \underbrace{\psi_1\sum_{i=1}^p \psi_iX_{t+1-i}}_{\in\;sp(X_1\...
1
Normally, if data were stationary and is with new data not stationary anymore one does not come to the conclusion that it will be so forever in the future. It may be that your time series may get back to its original stationary process, but only after a certain time (lag) that would lead to a model correction factor. The question is, what model/method did ...
2
This looks like a self-study question, so I will give you a hint:
Start with $\hat X_{t+1}$ as a linear combination of $X_1,\dots,X_t$ using the definition of AR(p).
Then do $\hat X_{t+2}$ as a linear combination of $X_1,\dots,X_{t+1}$ in the same way.
Replace $X_{t+1}$ in the previous step with $\hat X_{t+1}$ from the first step.
1
First off, here is a recreation of your plot with a black body radiation palette, which makes the temporal dynamics a bit easier to see (all R code at the bottom):
This does seem to support your thoughts. Here are time series of the estimated parameters per year:
The means seem to have a clear downward trend. The standard deviations and the degrees of ...
2
Most likely, seasonality is present at the lower levels. It's just that the signal to noise ratio is much lower at these levels. Aggregation improves this ratio to the point where seasonality becomes detectable.
A corollary: even if seasonality is present at lower levels, you may get better forecasts with non-seasonal models. These will be biased, but have ...
Top 50 recent answers are included
