0
$\begingroup$

I have run a linear regression with OLS for the period 2009 - 2017 and then complete back testing

The model is :

y=1.0527x - 0.082

Where

y= IOS (percentages) x = Bank Rate (percentages)

When i plot the y against x (with the actual data) and add the regression line it looks that the line doesn't fit the data - see the graph in the bottom. This is, i suppose, due to the outliers which impact R2 and coefficient

However, when we see the back testing (yellow line in the top diagram) the model OIS (2009 -2017) almost perfectly fits the data which i am struggled to understand or explain?

If the model is not correct how can correctly predict both periods and especially the 2009 - 2017 (top graph)?

My first thought is that since the Bank Rate is stable we might have violation of OLS assumptions but again how the back testing is almost correct?

Thanks,

Antonis

enter image description here enter image description here

$\endgroup$
5
  • $\begingroup$ (i cant comment (and i feel a little why to help/bad return so use it in good way why you dont use "yellow" model? or why you dont use 2000-2017 model :-) $\endgroup$
    – user2120
    Mar 8, 2021 at 20:32
  • $\begingroup$ let me explain, all this is part of a case study that i was asked to review and evaluate. What i asked above is part of the questions that i have to answer :) .Now, based on my understanding the model y=1.0527x - 0.082 was built using the 2009 - 2017 data and then it was used in the full period to see if it fits the data for 2000 - 2017 which actually it does and this is what i can't understand - i am sorry if i don't explain something well $\endgroup$
    – Ant
    Mar 8, 2021 at 21:03
  • 1
    $\begingroup$ all rates are highly correlated. OIS is obviously going to be highly correlated to whatever you pulled from the bank rate web site, and they have many different rate quotes. so the best thing for you is really just a constant spread. $\endgroup$
    – Aksakal
    Mar 8, 2021 at 21:27
  • 1
    $\begingroup$ I’m voting to close this question because it belongs to quant.stackexchange.com $\endgroup$
    – Aksakal
    Mar 8, 2021 at 21:28
  • $\begingroup$ Thank you for your comments but I am not very sure that I can understand what you mean? $\endgroup$
    – Ant
    Mar 8, 2021 at 22:31

1 Answer 1

0
$\begingroup$

i can comment only to "this answer"

you never know results before testing .

yes model fitted by data from 2009-2017 have better fit over whole dataset.

i guess red model, have "small" coefficient a1 ... y=intercept+a1*x because of that, in 2009-2017 "fit" is above it, its mainly intercept (and because data are moving less).

What is the main question? :-)

edit1. sorry i have to sleep. but you mention everything right about outlayer impact and R^2 formula too, i guess thats answer to question, to "explaing reasoning"

edit2: The main question is to evaluate the 2 models and which can be used for forecast the OIS?

  1. The main question is to evaluate the 2 models personaly, most simple way of evaluate models, are information criteria like R^2, AIC, RMSE, MAE - Model performance section at http://www.sthda.com/english/articles/38-regression-model-validation/158-regression-model-accuracy-metrics-r-square-aic-bic-cp-and-more/ Alpha for R^2 is usually 0.05= 5% "accuracy", so model is rejected as 0.4391 , considering accepted 0.95 value or higher. As you have only 1 metric for 1 model, you suggests not to use this model because it X doesnt explain Y at appriote alpha. As you dont have data, you can mention metrics (website) and suggest need for them-then you can give more conclusion.
  2. which can be used for forecast the OIS? None. Answer is not to using these models (because yellow have poor R^2 and red didnt has been backtested (metrics missing). Period. You can note, from empirical perspective ("method i look i see", sum of abs(residuals)= sum of absolute values of errors.. sum(abs(y-yPredicted))...by eye, it seems yellow model has smaller sum of absolute errors (then) < red model sum...considering criterium/metric "sum of abs errors", yellow model has better performance(lower value) but only with "i see" precission (for higher precision, numbers needed)

But write (collect) everything in this page, because your research/answer should mention all details (imho) :-)

Edit3 why yellow models isnt much bad according last paragraph at Definitions https://en.wikipedia.org/wiki/Coefficient_of_determination#Definitions

  • in the best case, the modeled values exactly match the observed values, which results R^2=1
  • A baseline model, which always predicts mean(y) will have R^2=0
  • Models that have worse predictions than this baseline will have a negative R^2

Your model have R^2 circa 0.4 which is a much much better then model which predict= mean(y) . It is "bad" model for alpha 0.05. for alpha 0.60 (1-alpha=0.40) it is acceptable model.

Thats why you can suggest not to use yellow model, because for alpha level 0.05% accuracy/precision of model's prediction have "higher error" then 5%.. (take care, information criteria like R^2 has complicated math, i used 5% error to simplify

(imagine you predict bank rate and you have model R^2 = 0.94 .. but customer/manager need alpha=0.01 .. or you have to fill cup with water, cup has capacity 0.5 liter. your hands shaking circa 0.1 liter .. you have high accuracy, but can you repeatly fill cup at 0.45? no, because your error will cause water to go out of cup (wanted 0.45 + 0.1 shaking = 0.55 but capacity was 0.5

yellow model is accepted for alpha 0.6. for alpha 0.5 is rejected.

hehe, but now you can add to your answer, that yellow model outperform "model which predict mean of y".

$\endgroup$
4
  • $\begingroup$ Thanks again. You are right the red model has smaller coefficient and it is built with the data from 2000 - 2008 and then used to forecast the 2000 - 2019. The main question is to evaluate the 2 models and which can be used for forecast the OIS? $\endgroup$
    – Ant
    Mar 8, 2021 at 22:43
  • $\begingroup$ Information criteria(s) can be used for comparing models. Post updated :-) $\endgroup$
    – user2120
    Mar 9, 2021 at 12:50
  • $\begingroup$ Thank you for the above - it is really useful. However, I am not very sure that i can understand how a "bad" model predicts well the period 2000 - 2017? this is something that is "real" (i mean we can see it in the graph). $\endgroup$
    – Ant
    Mar 9, 2021 at 20:13
  • $\begingroup$ @Ant i did edit 3. yes, for alpha=0.60 you can accept model with R^2=0.4391 :-) $\endgroup$
    – user2120
    Mar 9, 2021 at 20:51

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.