I finished watched Quantopian's Lecture on Kalman Filters and went through the notebook.

For those who want to learn more, I found the following links extremely useful:

The Python library that is being used is pykalman.

The code

In the Quantopian notebook, the meat of the code is here:

start = '2012-01-01'
end = '2015-01-01'
y = get_pricing('AMZN', fields='price', start_date=start, end_date=end)
x = get_pricing('SPY', fields='price', start_date=start, end_date=end)

delta = 1e-3
trans_cov = delta / (1 - delta) * np.eye(2) # How much random walk wiggles
obs_mat = np.expand_dims(np.vstack([[x], [np.ones(len(x))]]).T, axis=1)

kf = KalmanFilter(n_dim_obs=1, n_dim_state=2, # y is 1-dimensional, (alpha, beta) is 2-dimensional
                  initial_state_covariance=np.ones((2, 2)),

# Use the observations y to get running estimates and errors for the state parameters
state_means, state_covs = kf.filter(y.values)

Question 1: How to pick delta? Why delta / (1 - delta) * np.eye(2)?

Where does a delta of 1e-3 come from? And why not just do:

trans_cov = delta * np.eye(2) # How much random walk wiggles

Perhaps this is something that must be optimized using some cross-validation, although I'm not sure what metric to use. If anyone has any insight, would greatly appreciate it.

Question 2: How is observation_covariance = 2 decided?

I understand we're talk abouting prices here, and $2 move for a stock 'feels' like a good estimate for the variance of the price of Amazon, but is there a better way to select this rather than gut?

Question 3: Why is this approach better than just doing some rolling beta?

At the end of the day, to do some rolling beta you must decide what the lookback window is?

I understand the appeal of Kalman because you don't decide the lookback window, but you do need to decide transition_covariance? And changing this can have the same effect as increasing/decreasing your lookback window.

Question 4: Best practices

Are there best practices that I should be aware of? The only thing I can find was this on Quora.

Any help would greatly be appreciated. Feel free to leave me a comment if you have any questions for me.


closed as too broad by Juho Kokkala, Michael Chernick, John, mdewey, Nick Cox Jul 2 '17 at 18:55

Please edit the question to limit it to a specific problem with enough detail to identify an adequate answer. Avoid asking multiple distinct questions at once. See the How to Ask page for help clarifying this question. If this question can be reworded to fit the rules in the help center, please edit the question.

  • $\begingroup$ I also asked this question here. There are a few responses. quantopian.com/posts/… $\endgroup$ – JPN Aug 17 '15 at 12:21