Episode #125 of the Stack Overflow podcast is here. We talk Tilde Club and mechanical keyboards. Listen now

Questions tagged [forecasting]

Prediction of the future events. It is a special case of [prediction], in the context of [time-series].

Filter by
Sorted by
Tagged with
207
votes
3answers
19k views

How to know that your machine learning problem is hopeless?

Imagine a standard machine-learning scenario: You are confronted with a large multivariate dataset and you have a pretty blurry understanding of it. What you need to do is to make predictions ...
37
votes
2answers
7k views

Is it unusual for the MEAN to outperform ARIMA?

I recently applied a range of forecasting methods (MEAN, RWF, ETS, ARIMA and MLPs) and found that MEAN did surprisingly well. (MEAN: where all future predictions are predicted as been equal to the ...
35
votes
6answers
30k views

Best method for short time-series

I have a question related to modeling short time-series. It is not a question if to model them, but how. What method would you recommend for modeling (very) short time-series (say of length $T \leq 20$...
15
votes
3answers
36k views

Why use a certain measure of forecast error (e.g. MAD) as opposed to another (e.g. MSE)?

MAD = Mean Absolute Deviation MSE = Mean Squared Error I've seen suggestions from various places that MSE is used despite some undesirable qualities (e.g. http://www.stat.nus.edu.sg/~staxyc/T12.pdf, ...
5
votes
2answers
842 views

Simple method of forecasting number of guests given current and historical data

I am trying to predict the number of guests a restaurant might serve in a meal period based on the volume of business that same day from prior years (3-5 years of data), trends for the same day of the ...
9
votes
3answers
5k views

Transfer function in forecasting models - interpretation

I am occupied with ARIMA modelling augmented with exogenous variables for promotional modelling purposes and i have hard time explaining it to business users. In some cases software packages end up ...
19
votes
2answers
3k views

Why does minimizing the MAE lead to forecasting the median and not the mean?

From the Forecasting: Principles and Practice textbook by Rob J Hyndman and George Athanasopoulos, specifically the section on accuracy measurement: A forecast method that minimizes the MAE will ...
5
votes
3answers
16k views

Log or square-root transformation for ARIMA

With the below dataset, I have a series which needs transforming. Easy enough. However, how do you decide which of the SQRT or LOG transformations is better? And how do you draw that conclusion? <...
35
votes
1answer
40k views

Detecting Outliers in Time Series (LS/AO/TC) using tsoutliers package in R. How to represent outliers in equation format?

Comments: Firstly I would like to say a big thank you to the author of the new tsoutliers package which implements Chen and Liu's time series outlier detection which was published in the Journal of ...
16
votes
2answers
3k views

Is it possible to automate time series forecasting?

I would like to build an algorithm that would be able to analyze any time series and "automatically" choose the best traditional/statiscal forecasting method (and its parameters) for the analyzed time ...
68
votes
10answers
46k views

What is wrong with extrapolation?

I remember sitting in stats courses as an undergrad hearing about why extrapolation was a bad idea. Furthermore, there are a variety of sources online which comment on this. There's also a mention of ...
15
votes
2answers
27k views

Estimate ARMA coefficients through ACF and PACF inspection

How do you estimate the appropriate forecast model for a time series by visual inspection of the ACF and PACF plots? Which one (i.e., ACF or PACF) tells the AR or the MA (or do they both)? Which part ...
21
votes
2answers
20k views

Interpretation of mean absolute scaled error (MASE)

Mean absolute scaled error (MASE) is a measure of forecast accuracy proposed by Koehler & Hyndman (2006). $$MASE=\frac{MAE}{MAE_{in-sample, \, naive}}$$ where $MAE$ is the mean absolute error ...
83
votes
1answer
85k views

How to apply Neural Network to time series forecasting?

I'm new to machine learning, and I have been trying to figure out how to apply neural network to time series forecasting. I have found resource related to my query, but I seem to still be a bit lost. ...
18
votes
2answers
27k views

VAR forecasting methodology

I am building a VAR model to forecast the price of an asset and would like to know whether my method is statistically sound, whether the tests I have included are relevant and if more are needed to ...
11
votes
2answers
12k views

Forecasting hourly time series with daily, weekly & annual periodicity

Major edit: I would like to say big thanks to Dave & Nick so far for their responses. The good news is that I got the loop to work (principle borrowed from Prof. Hydnman's post on batch ...
5
votes
1answer
4k views

R Time Series Forecasting: Questions regarding my output

I'm working on a forecast for the following data: ...
6
votes
1answer
2k views

Why does default auto.arima stop at (5,2,5)?

The function auto.arima in the forecast package of R is a powerful tool to identify the best ...
5
votes
2answers
426 views

How to predict the next number in a series while having additional series of data that might affect it?

Let's say we want to predict the price of Big Mac for the year 2020. We have 2 indexes that we think might make an influence to Big Mac price determination. ...
16
votes
2answers
15k views

stochastic vs deterministic trend/seasonality in time series forecasting

I have moderate background in time series forecasting. I have looked at several forecasting books, and I don't see the following questions addressed in any of them. I have two questions: How would I ...
12
votes
1answer
2k views

Can you compare AIC values as long as the models are based on the same dataset?

I am doing some forecasting in R using Rob Hyndman's forecast package. The paper belonging to the package can be found here. In the paper, after explaining the automatic forecasting algorithms, the ...
16
votes
2answers
8k views

Getting started with neural networks for forecasting

I need some resources to get started on using neural networks for time series forecasting. I am wary of implementing some paper and then finding out that they have greatly over stated the potential of ...
15
votes
2answers
6k views

ARIMA estimation by hand

I'm trying to understand how the parameters are estimated in ARIMA modeling/Box Jenkins (BJ). Unfortunately none of the books that I have encountered describes the estimation procedure such as Log-...
6
votes
1answer
12k views

Explain the croston method of R

I am using crost() function of R for analyzing and forecasting intermittent demand/slow moving items time series. I am having difficulty in understanding the output....
6
votes
3answers
8k views

Does ARIMA require normally distributed errors or normally distributed input data?

I have two questions related to time series forecasting with ARIMA: Does ARIMA require normally distributed errors or normally distributed input data ? Are there any assumptions on input time series ...
6
votes
4answers
9k views

forecast using arima models [closed]

I am trying to predict values using arima(0,1,1). After doing predict(mod,n.ahead=5) (in R) am getting the same value for all ...
6
votes
3answers
19k views

Seasonality not taken account of in `auto.arima()`

I am having basically the same issue than in this thread, except one thing: The difference, in my case, is that my data is measured weekly and not daily, so the argument of a too high seasonality (> ...
24
votes
4answers
39k views

When to log transform a time series before fitting an ARIMA model

I have previously used forecast pro to forecast univariate time series, but am switching my workflow over to R. The forecast package for R contains a lot of useful functions, but one thing it doesn't ...
13
votes
2answers
10k views

Timeseries analysis procedure and methods using R

I am working on a small project where we are trying to predict the prices of commodities (Oil, Aluminium, Tin, etc.) for the next 6 months. I have 12 such variables to predict and I have data from Apr,...
18
votes
1answer
3k views

Paradox in model selection (AIC, BIC, to explain or to predict?)

Having read Galit Shmueli's "To Explain or to Predict" (2010) I am puzzled by an apparent contradiction. There are three premises, AIC- versus BIC-based model choice (end of p. 300 - start of p. 301):...
25
votes
2answers
997 views

When is it appropriate to use an improper scoring rule?

Merkle & Steyvers (2013) write: To formally define a proper scoring rule, let $f$ be a probabilistic forecast of a Bernoulli trial $d$ with true success probability $p$. Proper scoring ...
15
votes
1answer
10k views

Time Series Forecasting with Daily Data: ARIMA with regressor

I'm using a daily time series of sales data that contains about 2 years of daily data points. Based on some of the online-tutorials / examples I tried to identify the seasonality in the data. It seems ...
6
votes
1answer
11k views

Forecasting a seasonal time series in R

Forecasting airline passengers seasonal time series using auto.arima() I am trying to model some airline data in an attempt to provide an accurate monthly forecast ...
12
votes
4answers
5k views

Forecasting binary time series

I have a binary time series with 1 when the car is not moving, and 0 when the car is moving. I want to make a forecast for a time horizon up to 36 hours ahead and for each hour. My first approach ...
4
votes
1answer
3k views

stl() gives seasonal component, but ets() and auto.arima() choose nonseasonal models

I'm completely new to forecasting so please correct me if I'm wrong. I'm trying to forecast sales data using R. My main concern is that when I decompose the data using ...
4
votes
1answer
111 views

Why is removing instationarities a good thing when trying to forecast a time series?

Most introductory texts or tutorials to time-series forecasting mention that one should de-trend and de-seasonalize a time series first so that it becomes stationary. It is then easier to forecast ...
10
votes
4answers
4k views

Assessing forecastability of time series

Suppose i have a little over 20.000 monthly time series spanning from Jan'05 to Dec'11. Each of these representing global sales data for a different product. What if, instead of computing forecasts ...
10
votes
3answers
20k views

How do I handle nonexistent or missing data?

I tried a forecasting method and want to check if my method is correct or not. My study is comparing different kinds of mutual funds. I want to use the GCC index as a benchmark for one of them but ...
4
votes
3answers
2k views

Forecasting time series based on a behavior of other one

Apologies for this vague and unclear question, I have no background in statistics. I have two vectors of time series data, covering a six month period. The data is in daily intervals (except for ...
7
votes
1answer
2k views

Cross-validation for timeseries data with regression

I am familiar with "regular" cross-validation, but now I want to make timeseries predictions while using cross-validation with a simple linear regression function. I write down a simple example, to ...
4
votes
2answers
3k views

Prediction with GLS

Let's say I build a Generalized Least Squares model. I follow the standard procedure and first estimate a LM model. Then I create an error-response covariance matrix based on the residuals of this ...
7
votes
1answer
2k views

Why not using the R squared to measure forecast accuracy?

Why in literature usually the common accuracy measures like MAD, MSE, RMSE, MAPE ... are used. Why not using the $R^2$ (coefficient of determination)? I was thinking about the difference: By using ...
5
votes
2answers
2k views

How to use Dynamic Regression models in R to forecast future sales

I want to forecast the sales having 2 independent variables, x1 and x2. I want to choose between different combinations and lags, e.g: sales ~ x1 sales ~ lag(x1,-1) sales ~ lag(x1,-1) + lag(x2,-1) ...
3
votes
1answer
4k views

Forecasting hierarchical time series R package

I have to forecast a large set of (hierarchical) time series and since the R package hts allows for confidence intervals for their ensemble, I'd like to use it. I haven't found an example of how to ...
1
vote
2answers
13k views

Forecasting technique for daily data with monthly and day of week seasonality

I have daily data for 3 years. This sales data is of seasonal nature as business has spikes and downfall by month. Also, sales differ by each day of the week. for example, monday in general in a month ...
2
votes
2answers
157 views

Forecasting data with multiple seasonality

I'm attempting to forecast the number of taxi rides per hour that occur in NYC. I've turned the data into a time series using 24*7 as the frequency: ...
47
votes
3answers
3k views

AIC,BIC,CIC,DIC,EIC,FIC,GIC,HIC,IIC — Can I use them interchangeably?

On p. 34 of his PRNN Brian Ripley comments that "The AIC was named by Akaike (1974) as 'An Information Criterion' although it seems commonly believed that the A stands for Akaike". Indeed, when ...
37
votes
4answers
73k views

Difference between forecast and prediction?

I was wondering what difference and relation are between forecast and prediction? Especially in time series and regression? For example, am I correct that: In time series, forecasting seems to mean ...
9
votes
2answers
8k views

How to interpret and do forecasting using tsoutliers package and auto.arima

I have got monthly data from 1993 to 2015 and would like to do forecasting on these data. I used tsoutliers package to detect the outliers, but I do not know how do I continue to forecast with my set ...
16
votes
3answers
16k views

ETS() function, how to avoid forecast not in line with historical data?

I am working on an alogorithm in R to automatize a monthly forecast calculation. I am using, among others, the ets() function from the forecast package to calculate forecast. It is working very well. ...