1
$\begingroup$

I have a time series of discrete observations $X_s$. In total there are N observations. I have some assumption about the underlying data-generating process (it is a Markov process) and thus I know the density of the model $f(X_{s+1}|X_s,\theta)$ for model parameters $\theta$. Actually, I assume that there are two different processes with different parameters $\theta_1$ and $\theta_2$ such I have the densities $f(X_{s+1}|X_s,\theta_1)$ and $f(X_{s+1}|X_s,\theta_2)$. I assume that both processes take turns such that in some periods, the first process generates the data and in other periods the second one does it (I have concrete assumptions about when this should happen; sometimes a direct alternation is possible). I want to estimate the parameters $\theta_1$ and $\theta_2$ via maximum likelihood.

My idea therefore is: $$ \max \sum_{s=1}^{i=N} \log f(X_{s+1}|X_s,\theta) = \max \left[\sum_{i \in Q_1} \log f(X_{i+1}|X_i,\theta_1) + \sum_{j \in Q_2} \log f(X_{j+1}|X_j,\theta_2)\right]$$

where $Q_1$ and $Q_2$ are disjunct subsets of {1,...,N}. To get the parameters, I now maximize $$\max \sum_{i \in Q_1} \log f(X_{i+1}|X_i,\theta_1)$$ and $$\max \sum_{j \in Q_2} \log f(X_{j+1}|X_j,\theta_2)$$ separately such that I obtain $\theta_1$ and $\theta_2$. Is my proceeding sensible and mathematically correct? Or do I harm some preconditions? Thank you very much for your answer!

$\endgroup$
1
  • 2
    $\begingroup$ Your approach is correct provided $Q_1$ and $Q_2$ are known. If they are not known, you will need to use an EM algorithm. $\endgroup$
    – AdamO
    Commented Jul 21, 2017 at 19:57

1 Answer 1

1
$\begingroup$

If $Q_1$ and $Q_2$ are known, it can be treated as two separate problems, the way you do. Actually it is twice the same problem.

Otherwise it is a latent variable problem: whether an observation belongs to $Q_1$ or $Q_2$ is not observable. The "belonging" is latent.

The classical way to solve latent variable problem is the Expectation Maximization algorithm: https://en.wikipedia.org/wiki/Expectation%E2%80%93maximization_algorithm

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.