How to write down the log-likelihood expression for this moving average model

Question

Question is based on the paper Maximum likelihood for blind separation and deconvolution of noisy signals using mixture models pdf download link

Assume the model is a FIR system of order 2 expressed as

$$y(n) = \sum_{m=0}^M \theta_mx(n-m) + v(n) \tag{1}$$.

The model can be rewritten as $$y(n) = \mathbf{\theta^T}x(n) + v(n) \tag{2}$$

The state space representation would be: $$y(n) = \theta_0 x(n) + \theta_1 x(n-1) + \theta_2 x(n-2) \tag{3}$$, and

$$ y(n) = \theta^T x(n) + v(n) \tag{4} $$

Assuming the signal $x(n)$ is an $i.i.d.$ sequence of $(n \times 1)$ random vectors with independent components. $v(n)$ is an $i.i.d$ additive zero-mean Gaussian noise of unknown variance, $\sigma^2_v$. Here it still counts as a state space model, because $x(n)$ is still (albeit trivially) markov.

Question : Problem formulation : $\{x(n)\}$ (n by 1) is the unobserved input and $\{v(n)\}$ is the unobserved additive noise. The MLE solution will based on Expectation Maximization (EM). In the paper, I cannot understand how the likelihood expression would look like. Why do the Authors say mixture models?

What will be the complete likelihood and log-likelihood expression ?

This is how I proceeded.

I have used that fact that sum of 2 independent Gaussians is, again, Gaussian: $\theta^T x(n-M:n) + v(n) \sim \mathcal{N}(\theta^T \mu_x, \theta^T \Sigma_x \theta + 1)$.

Now, $\theta^T x(n-M:n)$ is Gaussian, too: a linear transformation of a Gaussian vector is still Gaussian. $\theta^T x(n-M:n) \sim \mathcal{N}(\theta^T \mu_x, \theta^T \Sigma_x \theta)$.

The variance of $y(n)$ is affected by $\theta$.

PDF of the observations $\mathbf{y}$ conditioned on the data sequence $x$ is

$$f(\mathbf{y|x,\theta}) = {\left(\frac{1}{\sqrt{\left(2\pi \sigma^2_v\right)}}\right)}^N \exp\left[-\frac{{\left(y-\theta^T x\right)}^2}{2\sigma^2_v}\right] \tag{1}$$ PDF of the complete data $\xi = [x, y]^T$:

$$f(\xi|\theta) = f(y|x,\theta)f(x) \tag{3}$$

Answer 1

"In the paper, I cannot understand how the likelihood expression would look like."

If they are using EM to estimate the model, then they are not using the (incomplete) likelihood function $f(y|\theta)$. They are using (evaluating) the complete likelihood function, which is the joint density of all $y$s and all $x$s. To get this, you would have to multiply your density $f(y|x, \theta)$ by $f(x|\theta)$. Edit: you have it on your last line in a general form, but you didn't write it out specifically for this model.

That's the beauty of EM. You can maximize something that depends on unobserved data using a clever iterative procedure. This is not what the contribution of the paper is, however.

"Why do the Authors say mixture models?"

It's because the observed data, which you denote $y(n)$ (different from the paper), is a mixture of the unobserved inputs, which you denote $x(n)$ (this is also different notation than the paper uses). If the dimension of $x(n)$ is one, however, this name might be slightly misleading. In either case, the observed data is a linear transformation of the unobserved data, with some extra noise added on top.

Edit: here is the complete data likelihood, as per request, with your notation (sort of).

\begin{align*} f(y_{1:T},x_{1:T};\theta) &= f(y_{1:T}|x_{1:T};\theta)f(x_{1:T};\theta) \\ &= \prod_{t=2}^T f(y_t|x_t) f(x_t|x_{t-1}) f(x_1) \\ &= \prod_{t=2}^T f(y_t|x_t) f(x_t) f(x_1) \\ &= \prod_{t=1}^T f(y_t|x_t) f(x_t) \end{align*}

with $f(y_t|x_t) = \text{Normal}(\theta ^T x_t, \Sigma_v)$