56
$\begingroup$

I don't understand what exactly is the difference between "in-sample" and "out of sample" prediction? An in-sample forecast utilizes a subset of the available data to forecast values outside of the estimation period. An out of sample forecast instead uses all available data. Are these correct?

Very specifically is the following definition correct?

A within sample forecast utilizes a subset of the available data to forecast values outside of the estimation period and compare them to the corresponding known or actual outcomes. This is done to assess the ability of the model to forecast known values. For example, a within sample forecast from 1980 to 2015 might use data from 1980 to 2012 to estimate the model. Using this model, the forecaster would then predict values for 2013-2015 and compare the forecasted values to the actual known values. An out of sample forecast instead uses all available data in the sample to estimate a models. For the previous example, estimation would be performed over 1980-2015, and the forecast(s) would commence in 2016.

Improve this question
$\endgroup$
5
  • $\begingroup$ Could you provide some context? The answers you provide to your own question seem O.K., but terminology might be subject-specific. $\endgroup$
    – IWS
    Commented Feb 9, 2017 at 12:07
  • 2
    $\begingroup$ Where did you get those definitions from? $\endgroup$
    – gung - Reinstate Monica
    Commented Feb 9, 2017 at 12:08
  • 1
    $\begingroup$ In-sample is data that you know at the time of modell builing and that you use to build that model. Out-of-sample is data that was unseen and you only produce the prediction/forecast one it. Under most circumnstances the model will perform worse out-of-sample than in-sample where all parameters have been calibrated. $\endgroup$
    – Richi W
    Commented Feb 9, 2017 at 12:11
  • $\begingroup$ @IWS I added spesific question :) $\endgroup$
    – Engin YILMAZ
    Commented Feb 11, 2017 at 16:41
  • $\begingroup$ @Richard Please read new spesific question... $\endgroup$
    – Engin YILMAZ
    Commented Feb 11, 2017 at 16:42

5 Answers 5

Reset to default
73
$\begingroup$

By the "sample" it is meant the data sample that you are using to fit the model.

First - you have a sample
Second - you fit a model on the sample
Third - you can use the model for forecasting

If you are forecasting for an observation that was part of the data sample - it is in-sample forecast.

If you are forecasting for an observation that was not part of the data sample - it is out-of-sample forecast.

So the question you have to ask yourself is: Was the particular observation used for the model fitting or not ? If it was used for the model fitting, then the forecast of the observation is in-sample. Otherwise it is out-of-sample.

if you use data 1990-2013 to fit the model and then you forecast for 2011-2013, it's in-sample forecast. but if you only use 1990-2010 for fitting the model and then you forecast 2011-2013, then its out-of-sample forecast.

Improve this answer
$\endgroup$
6
  • $\begingroup$ We have sample from 1990 to 2013,, then we fit the model on the sample,then we forecast 2011-2013,,is this in-sample? or We have sample from 1990 to 2013, then we fit the model 1990 to 2010 on the sample , we forecast 2011-2013, is this out of sample? $\endgroup$
    – Engin YILMAZ
    Commented Feb 13, 2017 at 20:45
  • 1
    $\begingroup$ yes, if you use data 1990-2013 to fit the model and then you forecast for 2011-2013, it's in-sample forecast. but if you only use 1990-2010 for fitting the model and then you forecast 2011-2013, then its out-of-sample forecast. $\endgroup$
    – Branislav Cuchran
    Commented Feb 13, 2017 at 22:03
  • $\begingroup$ @BranislavCuchran Can you cite a reference for these definitions? $\endgroup$
    – mareoraft
    Commented Aug 31, 2020 at 1:20
  • $\begingroup$ Here is a reference for the concept of "out-of-sample". In the reference they refer to "out-of-sample error" which appears to be the error of an out-of-sample forecast. en.wikipedia.org/wiki/Generalization_error $\endgroup$
    – mareoraft
    Commented Sep 6, 2020 at 20:54
  • $\begingroup$ Is out of sample and out of distribution the same meaning? $\endgroup$
    – n0p
    Commented Apr 9, 2021 at 10:59
4
$\begingroup$

Suppose in your sample, you have a sequence of 10 data points. This data can be divided into two parts - e.g. first 7 data points for estimating the model parameters and next 3 data points to test the model performance. Using the fitted model, predictions made for the first 7 data points will be called in-sample forecast and same for last 3 data points will be called out of sample forecast. This is same as the idea of splitting the data into training set and validation set.

Improve this answer
$\endgroup$
4
$\begingroup$

The below diagram will help you understand the IN TIME and OUT OF TIME

enter image description here

Improve this answer
$\endgroup$
3
$\begingroup$

In-sample forecast is the process of formally evaluating the predictive capabilities of the models developed using observed data to see how effective the algorithms are in reproducing data. It is kind of similar to a training set in a machine learning algorithm and the out-of-sample is similar to the test set.

Improve this answer
$\endgroup$
1
  • $\begingroup$ you give a succinct explanation of in-sample forecasting- could you also provide the same for out of sample (i.e. a short explanation not just a comparison to test sets)? $\endgroup$
    – ReneBt
    Commented Oct 15, 2018 at 15:45
0
$\begingroup$

I consider the in-sample is used to construct a model. And out-of-sample means to exam the model which uses im-sample data. It just likes the data analysis training and test.

Improve this answer
$\endgroup$

Not the answer you're looking for? Browse other questions tagged or ask your own question.