12
$\begingroup$

I don't understand what exactly is the difference between "in-sample" and "out of sample" prediction? An in-sample forecast utilizes a subset of the available data to forecast values outside of the estimation period. An out of sample forecast instead uses all available data Are these correct ?

Very specifically is the following definition correct ?

A within sample forecast utilizes a subset of the available data to forecast values outside of the estimation period and compare them to the corresponding known or actual outcomes. This is done to assess the ability of the model to forecast known values. For example, a within sample forecast from 1980 to 2015 might use data from 1980 to 2012 to estimate the model. Using this model, the forecaster would then predict values for 2013-2015 and compare the forecasted values to the actual known values. An out of sample forecast instead uses all available data in the sample to estimate a models. For the previous example, estimation would be performed over 1980-2015, and the forecast(s) would commence in 2016.

$\endgroup$
  • $\begingroup$ Could you provide some context? The answers you provide to your own question seem O.K., but terminology might be subject-specific. $\endgroup$ – IWS Feb 9 '17 at 12:07
  • $\begingroup$ Where did you get those definitions from? $\endgroup$ – gung Feb 9 '17 at 12:08
  • $\begingroup$ In-sample is data that you know at the time of modell builing and that you use to build that model. Out-of-sample is data that was unseen and you only produce the prediction/forecast one it. Under most circumnstances the model will perform worse out-of-sample than in-sample where all parameters have been calibrated. $\endgroup$ – Ric Feb 9 '17 at 12:11
  • $\begingroup$ @IWS I added spesific question :) $\endgroup$ – Engin YILMAZ Feb 11 '17 at 16:41
  • $\begingroup$ @Richard Please read new spesific question... $\endgroup$ – Engin YILMAZ Feb 11 '17 at 16:42
25
$\begingroup$

By the "sample" it is meant the data sample that you are using to fit the model.

First - you have a sample
Second - you fit a model on the sample
Third - you can use the model for forecasting

If you are forecasting for an observation that was part of the data sample - it is in-sample forecast.

If you are forecasting for an observation that was not part of the data sample - it is out-of-sample forecast.

So the question you have to ask yourself is: Was the particular observation used for the model fitting or not ? If it was used for the model fitting, then the forecast of the observation is in-sample. Otherwise it is out-of-sample.

if you use data 1990-2013 to fit the model and then you forecast for 2011-2013, it's in-sample forecast. but if you only use 1990-2010 for fitting the model and then you forecast 2011-2013, then its out-of-sample forecast.

$\endgroup$
  • $\begingroup$ We have sample from 1990 to 2013,, then we fit the model on the sample,then we forecast 2011-2013,,is this in-sample? or We have sample from 1990 to 2013, then we fit the model 1990 to 2010 on the sample , we forecast 2011-2013, is this out of sample? $\endgroup$ – Engin YILMAZ Feb 13 '17 at 20:45
  • $\begingroup$ yes, if you use data 1990-2013 to fit the model and then you forecast for 2011-2013, it's in-sample forecast. but if you only use 1990-2010 for fitting the model and then you forecast 2011-2013, then its out-of-sample forecast. $\endgroup$ – King Solomon's Horse Feb 13 '17 at 22:03
2
$\begingroup$

Suppose in your sample, you have a sequence of 10 data points. This data can be divided into two parts - e.g. first 7 data points for estimating the model parameters and next 3 data points to test the model performance. Using the fitted model, predictions made for the first 7 data points will be called in-sample forecast and same for last 3 data points will be called out of sample forecast. This is same as the idea of splitting the data into training set and validation set.

$\endgroup$
1
$\begingroup$

In-sample forecast is the process of formally evaluating the predictive capabilities of the models developed using observed data to see how effective the algorithms are in reproducing data. It is kind of similar to a training set in a machine learning algorithm and the out-of-sample is similar to the test set.

$\endgroup$
  • $\begingroup$ you give a succinct explanation of in-sample forecasting- could you also provide the same for out of sample (i.e. a short explanation not just a comparison to test sets)? $\endgroup$ – ReneBt Oct 15 '18 at 15:45

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.