I have a stationary dynamic system which at each given time $t$ is in state $x_t \in \mathcal{X}$. The set of states $\mathcal{X}$ is assumed to be finite but too large to be enumerated by a practical algorithm.

At some initial time $t$ I don't know the exact state but I know a probability distribution over the states, which I assume to be parametric and factorizable:
$\Pr[X_t=x] = q(x, \theta_t)$

At each step, the state is evolved by a (deterministic) state transition function: $x_{t+1} = f(x_t)$.
The problem is that after the state transition the posterior probability distribution over the states $p_{t+1}(x) = \sum_{x'} \Pr[X_{t+1}=x|X_t=x'] \cdot q(x', \theta_t)$ will be in general no longer representable by my parametric factorizable model $q(x, \theta)$.

I would like a method to find a parameter vector $\overset{*}{\theta_{t+1}}$ for my model that approximates the true distribution:
$\overset{*}{\theta_{t+1}} = argmin_{\theta} D[q(\cdot, \theta), p_{t+1}]$
where $D$ is some kind of dissimilarity measure (e.g. the Kullback–Leibler divergence).

I was thinking that some kind of variational method might be appropriate. I would like to avoid Monte Carlo methods if possible.

Any references or ideas on how to proceed?

  • $\begingroup$ Some more clarification may be needed: is the number of states finite, countably infinite or uncountable? Secondly, if you say that state transitions are deterministic, given by $x_{t+1}=f(x_t)$, why do you talk about $\mathrm{Pr}[X_{t+1}=x|X_t=x']$? Those probabilities are all either 0 or 1, no? Also, not sure if Kullback-Leibler is appropriate here since it is not symmetric ($D(a,b)\neq D(b,a)$). There are some modifications of the measure that are, however. $\endgroup$ – StijnDeVuyst Mar 24 '15 at 0:02
  • $\begingroup$ The number of states is finite, but too large to explicitly enumerate. The transition distribution is indeed deterministic. I mentioned the Kullback-Leibler divergence because, AFAIK, it is often used in variational Bayes methods, but any reasonable dissimilarity measure will be ok. $\endgroup$ – Antonio Valerio Miceli-Barone Mar 24 '15 at 16:41

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Browse other questions tagged or ask your own question.