# Tag Info

## Hot answers tagged state-space-models

42

A half-Cauchy is one of the symmetric halves of the Cauchy distribution (if unspecified, it is the right half that's intended): Since the area of the right half of a Cauchy is $\frac12$ the density must then be doubled. Hence the 2 in your pdf (though it's missing a $\frac{1}{\pi}$ as whuber noted in comments). The half-Cauchy has many properties; some are ...

17

Q1: What is the connection between PC time series and "maximum variance"? The data that they are analyzing are $\hat t$ data points for each of the $n$ neurons, so one can think about that as $\hat t$ data points in the $n$-dimensional space $\mathbb R^n$. It is "a cloud of points", so performing PCA amounts to finding directions of maximal variance, as you ...

11

Hamilton shows that this is a correct representation in the book, but the approach may seem a bit counterintuitive. Let me therefore first give a high-level answer that motivates his modeling choice and then elaborate a bit on his derivation. Motivation: As should become clear from reading Chapter 13, there are many ways to write a dynamic model in state ...

10

I did not see your question before. Yes, dynamic factor analysis can bee seen as a particular case of state-space model. It makes observations dependent of a small dimensional state vector (small relative to the dimension of the observation vector). So it is the same idea as in ordinary factor analysis, plus time dependence. The "factors" may have any time ...

10

You have $$y_t = x_t + v_t \tag{1}$$ and $$\phi(B)x_t = e_t.$$ Applying $\phi(B)$ to both sides of (1) yields \begin{align} \phi(B)y_t &= \phi(B)x_t + \phi(B) v_t \\ &= e_t + \phi(B) v_t. \tag{2} \end{align} Consider the right hand side of (2). This is clearly a covariance stationary process. By the Wold decomposition theorem it must have a ...

9

State-space models are very flexible; indeed they can encompass ARIMA models. One class of state space models that has some overlap with ARIMA models but also has a large subset of models that don't overlap with them is the Basic Structural Model (BSM). See Harvey (1989). There are also numerous papers by Harvey (usually with other authors) relating to ...

9

This is the same as above, but I thought I would provide a shorter, more concise answer. Again, this is Hamilton's representation for a causal ARMA($p$,$q$) process, where $r=\max(p,q+1)$. This $r$ number will be the dimension of the state vector $(\xi_t, \xi_{t-1},\ldots, \xi_{t-r+1})'$, and it is needed to make the number of rows of the state match up ...

9

Yes indeed: both exponential smoothing and ARIMA are special cases of state space models. For ARIMA, see this talk by Rob Hyndman, and for Exponential Smoothing, see Forecasting with Exponential Smoothing - the State Space Approach. This underlies the fact that specific Exponential Smoothing methods can be shown to yield MSE-optimal point forecasts for ...

8

They are not really different approaches in that they are solutions to different problems: one computes the sequence of filtering distributions $p(\beta_t|Y_{1:t})$, and the other the distributions based on all observations $p(\beta_t|Y_{1:T})$, for $t =1,...,T$. The smoother doesn't "hide underlying dynamics" but rather adjusts its state estimate (with ...

8

It is the case that the incomplete-data log likelihood has to increase at every step, but is not the case that the expected log likelihood has to increase at every step. The reason why is hidden in the fact that $\mathcal{Q}(\theta)$ should better be written $\mathcal{Q}(\theta^{t+1}; \theta^t)$. It is true that, as we are choosing the "best" $\theta^{t+1}$...

7

To me one of the main advantages is handling of missing data and uneven time steps. Kalman filter easily handles the missing observations, and actually can be used to impute them. OLS and MLE don't handle missing data as easily, and not every package will have this feature support unlike Kalman filter.

6

You could define the data generating process as follows: \begin{equation} x_{1,t} = \phi_{11} x_{1,t-1} + \cdots \phi_{1p} x_{1,t-p} + \epsilon_{1,t} \,, \quad \hbox{NID}(0, \sigma^2_1) \\ x_{2,t} = \phi_{21} x_{2,t-1} + \cdots \phi_{2p} x_{2,t-p} + \epsilon_{2,t} \,, \quad \hbox{NID}(0, \sigma^2_2) \\ \hbox{Cov}(\epsilon_{1,t}, \epsilon_{2,s}) = \sigma \, \...

6

The good news is that your instincts are right that it would be a useful technique. The bad news is that it's not a technique that you can use without understanding a fair amount of linear algebra. It's all about multiple equations with multiple matrix multiplications. Some tools like R's bsts package make it more accessible, but it's fundamentally more ...

5

Just to clarify one point to begin -- the components of the state and measurement vectors may themselves be in different units -- there's not a single unit for either the state nor the measurement (observation). Indeed even the dimension of either one could change over time. Note that: $\bar{P}$ is not in "units of the state". The units of its components ...

5

I personally think the question is too broad to be answered well, But I still want to give some suggestions. I feel Murphy's introduction to graphical models is very useful and it covers Bayesian Network with discrete time very well. If you have not checked this, I would recommend to read this first. A Brief Introduction to Graphical Models and Bayesian ...

5

I also have to speak regularly to people who do not have a technical background, and here is how I would approach it: First, unless your audience knows about the normal distribution, I would not even mention DLM, I would just talk about state space models. I would still give them a DLM set of equations as an example (linear is easy to understand), but I have ...

5

Answer: There is a mistake in the formula for $\theta$. The correct computation must align autocovariances of the MA components of two representations. The correct formula is $$\theta = \frac{\sqrt{\xi^2-4} -\xi}{2}$$ where $\xi:= \phi + \frac{\sigma^2_v+\sigma^2_w}{\phi \sigma^2_v}$. Substituting the chosen values for $\phi,\sigma_v,\sigma_w$ gives $\... 4 Correlation can be introduced via the covariance matrices, but also through non zero off-diagonal terms in the transition or observation matrix. 4 Some explicit guidance and hints: Your answer in (a) looks okay to me. In (b) you would either need to go on and show the properties of the series$x_t$(what's its ACF, say? What are the properties you need?) or to explicitly rewrite it in the form of an MA (which is easier, I think - you might just recast it as a transform to an MA in$\zeta$say,$x_t=\...

4

Instead of guessing its value, you should include $\lambda_c$ in the set of parameters to be estimated by means of some method or rule. For example, you can estimate the parameters by maximum likelihood. Upon the state-space representation of the model, the likelihood function can be evaluated by means of the Kalman filter. The likelihood function can be ...

4

I highly recommend that you read Rob Hyndman's free online text, Forecasting: principles and practice. More specifically, a reference that will be helpful to your specific question is discussed on this page of the text. As you can see, exponential smoothing models are non-stationary in nature, whereas ARIMA models can be stationary. Additionally, the ...

4

In my understanding you have to put restrictions on parameters, for example setting them to a constant, to ensure identification. There is no way to rewrite an unidentified model, while preserving all parameters, to a identified model. There is however an algorithm to check whether a SS-model is identified. Try looking up the article: J. Casals, A. ...

4

In principle, you could certainly use Gibbs sampling or Metropolis Hasting to take draws from the posterior of a state space model. There is nothing about these models that invalidates the theory that justifies either of these methods. In practice, it's my understanding that this is a bad idea. In short, the number of samples required would be unreasonable ...

4

Although I did not find this explicitly stated in the paper, it seems that $f$ is actually intended to be a multiple output Gaussian process (possibly with independent components) - so that a realization is a function from $\mathbb{R}^{n_x}$ to $\mathbb{R}^{n_x}$ and thus $\mathbb{f}_t = f(\mathbf{x}_t) \in \mathbb{R}^{n_x}$. Note that: In the first author'...

3

You have arrived to the stationary form of the local level model: $$\Delta y_t \equiv x_t = \underbrace{\Delta \alpha_t}_{\eta_{t-1}} + \Delta \epsilon_t \,,$$ where $\Delta$ is the difference operator such that $\Delta y_t = y_t - y_{t-1}$. Now, I think it is easier to first check the statistical properties (mean, covariances, autocorrelations) of this ...

3

If your goal is to forecast the variables, then the pseudo-out-of-sample analysis is a good way to choose the best model. Apparently that is what you've done between the ETS models, but you can add the ARIMA to the horse-race. You will be interested in analyzing some forecasting performance measures, such as RMSE and MAPE, and choosing the model that ...

3

The way this is done, is to first establish the relationship between $\alpha_{t}$ and $\alpha_{t}^{\ast}$ and proceed from there. We take the initial state equations above and take $$\alpha_{t}^{\ast} = \mathsf{T}_{t}^{-1}\mathsf{W}_{t}\beta + \alpha_{t},$$ we see that we can write \alpha_{t + 1}^{\ast} = \mathsf{T}_{t}\alpha_{t}^{\ast} + \mathsf{R}_{t}...

3

I think what you say is correct, and I do not think it is messy. A way of phrasing it would be to say that the Kalman filter is an error-correction algorithm, that modifies predictions in the light of the discrepancies with current observations. This correction is made in your step 4) using the gain matrix $A_t$.

3

Yes, it is possible with two states. Use dlmModPoly() to form a dlm from a second order polinomial. This initializes a model with the following parametrization: $y_t = \begin{pmatrix} 1 & 0 \end{pmatrix} \space \theta_t + \nu_t, \nu_t \sim N(0,V_t) \\ \theta_t = \begin{pmatrix} 1 & 1 \\ 0 & 1 \end{pmatrix} \theta_{t−1} + ω_t, ω_t \sim N(0, W_t)$...

3

You can cast and AR model in state-space form (and an ARMA model, or dynamic regression model). The Kalman filter is an algorithm that enables you to recursively compute the state vector (and the likelihood, with normal data): thus, it indirectly provides a way to maximize the likelihood.

Only top voted, non community-wiki answers of a minimum length are eligible