4
$\begingroup$

This question follows the one asked there.

I am trying to filter an equity index (Stoxx 600) time series using kalman filter. I'm using the R package dlm and my code is inspired from the dlm vignette with the Nile data filtering.

I was first trying to understand why my filtered data was similar to my observations and it turns out (as it has been said in the question mentioned above) that when estimating my model parameters, the variance of my measurement equation is extremely small (around 10e-5) compared to the one of the state equation (above 300). I am actually quite surprised by those results as I get similar ones for other financial time series (including macroeconomic ones) and those kind of time series are well known to be noisy.

Could you please explain to me if it is the model that I'm not specifying well or if it is the MLE fitting process that is not working correctly?

As in the Nile example of the dlm vignette, I'm using a random walk plus noise, with unknown system and observation variances.

Here's the code I'm using. The a variable is a vector with monthly historical data for the Stoxx 600 since 1999 :

rm(list = ls())

library(dlm)

a <- c(480.77,457.39,489.58,500.62,499.73,485.4,481.69,489.68,506.28,484.09,497.97,466.47,462.34,465.78,431.22,415.07,442.34,444.06,428.4,412.54,388.32,347.75,362.46,377.86,390,382.75,379.94,398.31,383.08,369.31,336.9,302.21,302.07,259.61,284.17,297.08,271.6,252.77,243.61,237.44,263.62,267.1,276.5,288.04,293.83,283.33,303.66,306.48,314.85,323.99,333.77,326.79,331.47,330.75,336.45,330.52,327.67,333.68,337.44,346.74,353.38,361.71,372.89,371.1,364.27,382.35,395.58,408.92,409.82,428.19,418.18,432.27,447.66,463.83,474.26,485.52,490.11,467.53,471.33,479.63,493.79,503.97,521.84,520.64,540.76,552.11,541.26,556.82,578.23,596.82,593.48,573.34,568.6,571.85,588.38,562.03,553.54,489.4,485.27,467.22,496.36,499.47,449.55,441.22,449.27,399.68,346.95,323.43,311.26,300.43,272.53,278.88,318.69,334.33,331.34,362.45,381.22,392,383.26,387.87,412.09,401.04,399.9,430.01,425.74,404.75,402.79,423.12,417.27,431.61,442.33,436.37,459.95,467.45,479,462.53,478.25,477.73,464.8,452.25,405.81,387.2,417.09,412.45,420.35,437.85,456.02,455.67,448.15,421.12,442.4,460.82,470.61,475.28,479.01,489.52,496.77,510.74,516.59,525.07,533.78,544.67,517.06,543.98,541.18,565.63,587.87,593.91,600.04,590.1,619.58,614.69,624.31,640.55,637.2,626.78,639.53,642.13,630.97,651.57,643.24,689.89,738.07,750.44,751.19,763.73,729.46,758.74,696.3,668.02,722.01,742.33,705,660.1,645.53,654.57,666.03)

data_to_fit <- a

buildFun <- function(x) {
  dlmModPoly(1, dV = exp(x[1]), dW = exp(x[2]))
}

fit <- dlmMLE(data_to_fit, parm = c(0,0), build = buildFun)

print(exp(fit$par[1]))
print(exp(fit$par[2]))

dlm_Jump <- buildFun(fit$par)

JumpFilt <- dlmFilter(data_to_fit, dlm_Jump)

plot(data_to_fit, type = 'o')
lines(dropFirst(JumpFilt$m), type = 'o', pch = 20, col = "brown")

Thanks

Improve this question
$\endgroup$
1
  • 1
    $\begingroup$ I cannot debug your code. However I can explain you what is happening as I tried to fit various Kalman models to stock prices in the past. Stock prices follow a random walk model. There is almost 0 observation noise, unless it is intraday tick data, in which case observation noise is the bid-ask oscillation (which is irrelevant for prediction purposes). Basically all your noise comes from the process itself which makes it very unpredictable based on historical price information. Basically when you observe a price it is actually the underlying state, hence there is nothing to filter. $\endgroup$
    – Cagdas Ozgenc
    Commented May 25, 2016 at 15:07

1 Answer 1

Reset to default
2
$\begingroup$

I think you should try to estimate first on returns and then re-build your level series with the filtered returns.

edit : I'll detail a little bit.

If you take level series with your specification of model (model that I'll call #1 : Random walk) you are trying to estimate the trend of a random walk hence the perfect fit.

  • 1) Random walk.

    $x=x_{(t-1)}+ε$

You can also use a deterministic trend including in a different model (#2) with a deterministic trend to estimate mu.

  • 2) Deterministic trend.

    $x=x_{(t-1)}+μ+ε$

Last, and what I would recommend, is to use a stochastic trend by using two state-space-variables in a third model (#3)

  • 3) Stochastic trend.

    $x=x_{(t-1)}+z+ε$
    $z=z_{(t-1)}+ε$

First answer was just an easy (and a bit fallacious too) way to get your first difference and should have also worked, didn't it?

Thierry Roncalli's paper for source : a link

Improve this answer
$\endgroup$
3
  • $\begingroup$ @ylnor, I tried to filter the returns and it worked. I suppose working on the returns removed the trend which allows my data to be filtered with a random walk. I also agree that adding a trend to my state equation should solve the problem too (I'll try it). Thanks $\endgroup$
    – Ben
    Commented May 25, 2016 at 16:26
  • $\begingroup$ @Ben It just appears to work. When you fit a Kalman model on the return series, the state will oscillate very close to 0, which will again be practically useless. Adding a trend to the original model will also make you estimate a hallucination. I did this thousand times for financial and economic data. The above recommendations are totally useless for your scenario. $\endgroup$
    – Cagdas Ozgenc
    Commented May 25, 2016 at 19:33
  • 1
    $\begingroup$ @CagdasOzgenc, I really don't see your point here, except saying you were not able to do it, you don't provide any information or improvement to the asked question. Could explain us the theory behind estimating 'hallucination' when using a stochastic trend ? Ben, Also, estimation of trends through Kalman filter is a recurrent topic in the financial theory, you will find many papers doing so. I would highly recommend Roncalli's paper (link ) , for quick read of the models, just go through pages 11-13. $\endgroup$
    – ylnor
    Commented May 26, 2016 at 8:40

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.