New answers tagged forecasting
0
Depends on the model you plan on using, i.e multiplicative or additive error. Please see table given below, which from Forecasting:Principles and Practice
The table is for multiplicative error, now for additive trend and multiplicative seasonality (set of equations in 3rd column and 2nd row) you can find the error equation by working backwards for the ...
0
Problem Framing:
Without the strategy part, this would be a straight up forecasting problem.
But once you add the concept of choosing among different strategies, it becomes an optimization problem, also referred to as a decision problem, not a predictive one: This is not about using independent inputs to predict an outcome of a dependent variable.
It is ...
0
Depending on what tool you are using for Holt you can always set the value for $\beta$ close to $0.9$ to make it more reactive to recent changes.
Keep in mind the formula for Holt's method:
$$\hat{Y}_{t+1} = \alpha Y_{t} + (1-\alpha)(\hat{L}_{t}+\hat{T}_{t})$$
$$\hat{T}_{t} = \beta (Y_{t}-Y_{t-1})+(1-\beta)\hat{T}_{t-1}$$
$Y_t$ is the actual value, $\hat{Y}_{...
1
I don't recall the exact source where I read this, but neither auto.arima nor pmdarima are really optimized to scale, which might explain the issues you are facing.
But there are some more important things to note about your question: With 80K data points at 15 minute intervals, ARIMA probably isn't the best type of model for your use case anyway:
With the ...
2
Well, let's think at what those numbers actually mean.
If we take feature g for example, we know that our model relies on it a bit. So much that if you shuffle its values when making predictions on train data, your model performance drops by around $0.002$ (if I am correct).
However, when we do the same thing and shuffle it before predicting unseen data, ...
0
"You acted unwisely," I cried, "as you see By the outcome." He calmly eyed me: "When choosing the course of my action," said he, "I had not the outcome to guide me."
(Ambrose Bierce: A Lacking Factor, from The Scrap Heap)
Great way to pose the question!
I'd like to offer a non-answer which presents the forecast ...
3
The difference in the observed importance of some features when running the feature importance algorithm on Train and Test sets might indicate a tendency of the model to overfit using these features. This is indeed closely related to your intuition on the noise issue.
In other words, your model is over-tuned w.r.t features c,d,f,g,I.
Running feature ...
3
First: ignore the results you have for the training set, they are worthless. Who cares how good a feature is at predicting for records that built the model?
Second: At this point you can’t do anything with features c,d,f,g. You’ve already built your model, yes these features are not useful in predicting the values for your test set, but if you were to remove ...
1
This is known as an ARMAX model. Note that this is different from what forecast::auto.arima() with external regressors provided in the xreg parameter fits, which is a regression with ARIMA errors. If you search for "ARMAX R", you need to be careful about the distinction between the two kinds of models, because this is very frequently confused. More ...
1
Here is a working example for the extraction of the desired distributional components. Bear in mind that this is the distribution on a transformed scale. Maybe this link gives some more explanation on this topic:
Forecasting using transformations
library(tsibbledata)
library(tsibble)
library(dplyr)
#>
#> Attache Paket: 'dplyr'
#> The following ...
0
To do the forecasting, you need to decide the parameters of your model because you do forecasting using the model.
And the model paramters are usually decided by maximum likelihood estimation(MLE). To do the MLE, you need to assume a distribution on the residuals($\varepsilon_t$) because likelihood is calculated from the probability distribution. And we ...
0
A little information regarding your first point on the selection of Prediction Intervals.
In general, forecasts are used as an input in a subsequent decision making process, usually an optimization problem. For example, given a set of stock price predictions, maximize portfolio returns. In that regard, it is common to make decisions that take risk under ...
3
Indeed, the procedure you describe is what it is typically done in mixed-effects models. When you fit the models under maximum-likelihood you only get $\hat \theta$, and then using empirical Bayes you get an estimate of $\hat b_i(\hat \theta)$, which you plug-in the equation to obtain a prediction for a particular subject. In the context of linear mixed ...
4
+1 to Gordon's answer.
Forecast accuracy "guidance" or "benchmarks" are not worth the bits they take up. They are typically derived from surveys on a convenience sample. I went into some detail in a critique of such benchmarks in an article (Kolassa, 2008, Foresight: The International Journal of Applied Forecasting). Yes, it's a couple of ...
4
Those Lewis numbers are fairly arbitrary, you cant just say that a 20% error is good forecasting because some guy wrote it in a book 40 years ago.
The acceptable margin or error completely depends on the problem domain. In some situations a model that gives a 20% error will be great, in others it will be unusable. I know its tempting to rely on general rules ...
1
Because it is the best ARIMA fit for your data, there is likely no mistake.
The 63rd observation is indeed a big outlier. It is ~30 times bigger than standard deviation for other observations. Even if there was some autocorrelation structure in your data, you would often not be able to discern it with such outlier, so trying to remove it is a good approach.
...
1
library(forecast)
fit <- Arima(USAccDeaths, order=c(0,1,1), seasonal=c(0,1,1))
fit %>% forecast() %>% autoplot()
Created on 2020-06-23 by the reprex package (v0.3.0)
0
I think ultimately how good your model is, is how well you predict the future (out of sample predictions). There is no "correct" answer to that, ultimately it is what you and/or those you report will accept. Its worth considering how well others do relative to your predictions. In some areas with high process uncertainty its difficult to be ...
0
Something along these lines, where you would of course probably also want to plot the original time series etc.
y <- arima.sim(list(order=c(1,0,0), ar=0.9), n=120)
y.is <- y[1:100] # in-sample und ...
y.oos <- y[101:120] # out-of-sample
reg.AR1 <- arima(y.is, order=c(1,0,0))
fore.AR1 <- predict(reg.AR1, n.ahead=20)$pred
plot(fore.AR1)
0
Note that auto.arima() fits a regression with ARIMA errors, not an ARIMAX model. There is a difference. Thus, your model is
$$ y_t=X_t\beta+\epsilon_t,$$
where $X_t$ contains your regressors at time $t$, and
$$ \epsilon_t\sim\text{ARIMA(1,1,0) with drift,} $$
or
$$ (\epsilon_t-\epsilon_{t-1}) = \gamma t+\phi(\epsilon_{t-1}-\epsilon_{t-2})+\eta_t.$$
$\gamma$ ...
3
The reference page on the fable website contains an organised list of models: https://fable.tidyverts.org/reference/
1
Your choice of Prediction Interval (PI) width depends on what you plan on doing with the PI. For instance, if you will use it to set safety stocks, you can determine the optimal quantile, be it 80%, 90% or 95%. (In this case, you would usually not use both endpoints of the PI, but only the upper one.)
Using a "100% PI" often does not make a lot of ...
0
I would first try some basic tests to ascertain if your methodology is producing any forecast of value.
For example, assume that there is a possible interest in a correct forecast greater (or less) than say k%. Tabulate the number of times the database indicated an expected forecast value greater than k% in total of n forecast cases. Compare this to the ...
0
I think this is a time series not r question. I think the norm is to use all the data to predict the future not the training data set.
1
The answer of rjt90 is correct but since I'm working on these models, I thought I expand on it.
Score-driven framework
In the class of score-driven models (or GAS models), the time-varying parameter $\alpha_t$ is updated over time using an autoregressive updating function based on the score of the conditional observation probability density function, see
...
1
I played with Prophet a bit. They promise big. As I understood their claim was for the framework for massive forecasting. If you have 10,000 series to forecast, there's no way to do it manually. So, let's just run the thing on all of them automatically, and maybe we'll get a decent set of forecast on average.
In finance we also forecast massive numbers of ...
5
ARIMA and similar models assume some sort of causal relationship between past values and past errors and future values of the time series: $$Y_{t+h}=f(Y_{t},Y_{t-1},Y_{t-2},....,\epsilon_{t},\epsilon_{t-1},\epsilon_{t-2},...)$$ e.g. the volatility of a stock today is causally driven by the volatility of that stock yesterday and two days ago, the population ...
3
The most reliable way of understanding the capabilities of a package is, as always its CRAN page. In the specific case of the fable package, we find its reference manual and two different vignettes, one introduction and one vignette on forecasting with transformations.
The reference manual in particular looks helpful. For instance, I see no less than ten ...
1
Do you think it is a good way to solve this problem or should I re-construct my architecture? I wonder if there are other proven ways for SKU level price optimization in an e-com scenario. Any ideas, inputs would be highly appreciated.
In theory, you can use an MLP for just about anything (thanks to the universal approximation theorem) so your approach is ...
0
I don't think you can actually forecast covid19 at this point. There is too little data and too little is known about the process which, since it is a new disease, is likely to change anyhow (a structural break I guess you could see this as). A second issue are your questions on arima - which there really are too many to answer at one time. You ask for ...
0
I do this for a living and so its important to me.:) However, I am a data analyst rather than a statistician so my answer might vary from a statistician's. The way I assess it if my model is good enough is two fold. First, I track a percent difference in each month and a year to date (the year is what really matters to us). My rule of thumb is that five ...
1
There are multiple issues present.
The first one is that otm.arxiv() does not follow the standard R practice of returning a fitted model that one applies forecast() to. Instead, it performs fitting and forecasting. To obtain a forecast from otm.arxiv(), you need to supply an h parameter to it:
otm.arxiv(y,h=3,thetaList=seq(from=1,to=5,by=0.5),g="SE&...
1
Given your model, your forecast is reasonable whichever of the two ways you specify the external regressor (I think the two ways are equivalent; note that the true value of $\mu$ is defined relative to how you specify the external regressor). You just extrapolate the negative trend, having adjusted for the mean. The mean is high relative to the few preceding ...
Top 50 recent answers are included
