36
$\begingroup$

I don't understand what exactly is the difference between "in-sample" and "out of sample" prediction? An in-sample forecast utilizes a subset of the available data to forecast values outside of the estimation period. An out of sample forecast instead uses all available data Are these correct ?

Very specifically is the following definition correct ?

A within sample forecast utilizes a subset of the available data to forecast values outside of the estimation period and compare them to the corresponding known or actual outcomes. This is done to assess the ability of the model to forecast known values. For example, a within sample forecast from 1980 to 2015 might use data from 1980 to 2012 to estimate the model. Using this model, the forecaster would then predict values for 2013-2015 and compare the forecasted values to the actual known values. An out of sample forecast instead uses all available data in the sample to estimate a models. For the previous example, estimation would be performed over 1980-2015, and the forecast(s) would commence in 2016.

$\endgroup$
5
  • $\begingroup$ Could you provide some context? The answers you provide to your own question seem O.K., but terminology might be subject-specific. $\endgroup$
    – IWS
    Feb 9 '17 at 12:07
  • 1
    $\begingroup$ Where did you get those definitions from? $\endgroup$ Feb 9 '17 at 12:08
  • $\begingroup$ In-sample is data that you know at the time of modell builing and that you use to build that model. Out-of-sample is data that was unseen and you only produce the prediction/forecast one it. Under most circumnstances the model will perform worse out-of-sample than in-sample where all parameters have been calibrated. $\endgroup$
    – Ric
    Feb 9 '17 at 12:11
  • $\begingroup$ @IWS I added spesific question :) $\endgroup$ Feb 11 '17 at 16:41
  • $\begingroup$ @Richard Please read new spesific question... $\endgroup$ Feb 11 '17 at 16:42
55
$\begingroup$

By the "sample" it is meant the data sample that you are using to fit the model.

First - you have a sample
Second - you fit a model on the sample
Third - you can use the model for forecasting

If you are forecasting for an observation that was part of the data sample - it is in-sample forecast.

If you are forecasting for an observation that was not part of the data sample - it is out-of-sample forecast.

So the question you have to ask yourself is: Was the particular observation used for the model fitting or not ? If it was used for the model fitting, then the forecast of the observation is in-sample. Otherwise it is out-of-sample.

if you use data 1990-2013 to fit the model and then you forecast for 2011-2013, it's in-sample forecast. but if you only use 1990-2010 for fitting the model and then you forecast 2011-2013, then its out-of-sample forecast.

$\endgroup$
5
  • $\begingroup$ We have sample from 1990 to 2013,, then we fit the model on the sample,then we forecast 2011-2013,,is this in-sample? or We have sample from 1990 to 2013, then we fit the model 1990 to 2010 on the sample , we forecast 2011-2013, is this out of sample? $\endgroup$ Feb 13 '17 at 20:45
  • $\begingroup$ yes, if you use data 1990-2013 to fit the model and then you forecast for 2011-2013, it's in-sample forecast. but if you only use 1990-2010 for fitting the model and then you forecast 2011-2013, then its out-of-sample forecast. $\endgroup$ Feb 13 '17 at 22:03
  • $\begingroup$ @BranislavCuchran Can you cite a reference for these definitions? $\endgroup$
    – mareoraft
    Aug 31 '20 at 1:20
  • $\begingroup$ Here is a reference for the concept of "out-of-sample". In the reference they refer to "out-of-sample error" which appears to be the error of an out-of-sample forecast. en.wikipedia.org/wiki/Generalization_error $\endgroup$
    – mareoraft
    Sep 6 '20 at 20:54
  • $\begingroup$ Is out of sample and out of distribution the same meaning? $\endgroup$
    – n0p
    Apr 9 at 10:59
4
$\begingroup$

Suppose in your sample, you have a sequence of 10 data points. This data can be divided into two parts - e.g. first 7 data points for estimating the model parameters and next 3 data points to test the model performance. Using the fitted model, predictions made for the first 7 data points will be called in-sample forecast and same for last 3 data points will be called out of sample forecast. This is same as the idea of splitting the data into training set and validation set.

$\endgroup$
4
$\begingroup$

The below diagram will help you understand the IN TIME and OUT OF TIME

enter image description here

$\endgroup$
2
$\begingroup$

In-sample forecast is the process of formally evaluating the predictive capabilities of the models developed using observed data to see how effective the algorithms are in reproducing data. It is kind of similar to a training set in a machine learning algorithm and the out-of-sample is similar to the test set.

$\endgroup$
1
  • $\begingroup$ you give a succinct explanation of in-sample forecasting- could you also provide the same for out of sample (i.e. a short explanation not just a comparison to test sets)? $\endgroup$
    – ReneBt
    Oct 15 '18 at 15:45
0
$\begingroup$

I consider the in-sample is used to construct a model. And out-of-sample means to exam the model which uses im-sample data. It just likes the data analysis training and test.

$\endgroup$
-1
$\begingroup$

In Time series Forecasting, 'Insample' means train data 'Outsample' means test data

In Time series, first we can forecast results for 'Insample'(i.e. train) data. Later we can forecast the results for 'outsample' (i.e. test) data.

model = ARIMA(order = (p,d,q), seasonal_order=(P,D,Q,S))
model.fit(train_data)

train_predictions = model.predict_in_sample()
test_predictions = model.predict(n_periods=len(test_data.index))

predictions = pd.concatenate((train_predictions, test_predictions),axis=0)
$\endgroup$
2
  • 1
    $\begingroup$ I think your answer is getting downwoted, because it does not answer the question - in particular "Very specifically is the following definition correct ?" is not adressed. $\endgroup$ Sep 4 '19 at 14:57
  • $\begingroup$ @MartinModrák The answer is downvoted because it is wrong. Out-of-sample error (or risk) means the error given the (unknown) real probability distribution of the data. $\endgroup$
    – Julian L.
    Apr 7 '20 at 10:41

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.