I finished watched Quantopian's Lecture on Kalman Filters and went through the notebook.

For those who want to learn more, I found the following links extremely useful:

The Python library that is being used is pykalman.

The code

In the Quantopian notebook, the meat of the code is here:

start = '2012-01-01'
end = '2015-01-01'
y = get_pricing('AMZN', fields='price', start_date=start, end_date=end)
x = get_pricing('SPY', fields='price', start_date=start, end_date=end)

delta = 1e-3
trans_cov = delta / (1 - delta) * np.eye(2) # How much random walk wiggles
obs_mat = np.expand_dims(np.vstack([[x], [np.ones(len(x))]]).T, axis=1)

kf = KalmanFilter(n_dim_obs=1, n_dim_state=2, # y is 1-dimensional, (alpha, beta) is 2-dimensional
                  initial_state_covariance=np.ones((2, 2)),

# Use the observations y to get running estimates and errors for the state parameters
state_means, state_covs = kf.filter(y.values)

Question 1: How to pick delta? Why delta / (1 - delta) * np.eye(2)?

Where does a delta of 1e-3 come from? And why not just do:

trans_cov = delta * np.eye(2) # How much random walk wiggles

Perhaps this is something that must be optimized using some cross-validation, although I'm not sure what metric to use. If anyone has any insight, would greatly appreciate it.

Question 2: How is observation_covariance = 2 decided?

I understand we're talk abouting prices here, and $2 move for a stock 'feels' like a good estimate for the variance of the price of Amazon, but is there a better way to select this rather than gut?

Question 3: Why is this approach better than just doing some rolling beta?

At the end of the day, to do some rolling beta you must decide what the lookback window is?

I understand the appeal of Kalman because you don't decide the lookback window, but you do need to decide transition_covariance? And changing this can have the same effect as increasing/decreasing your lookback window.

Question 4: Best practices

Are there best practices that I should be aware of? The only thing I can find was this on Quora.

Any help would greatly be appreciated. Feel free to leave me a comment if you have any questions for me.

  • $\begingroup$ I also asked this question here. There are a few responses. quantopian.com/posts/… $\endgroup$ – JPN Aug 17 '15 at 12:21