Beginner learning resources : Pdf and likelihood function for non-Gaussian time series model

Question

I am struggling with exercise problems related to blind system identification where the knowledge about the source input is assumed to be known using maximum likelihood estimation of univariate time series processes which are non-Gaussian. For example, an AR model that is excited by non-Gaussian distribution like Bernoulli and then estimation from noisy time series where the process noise is Gaussian.

The AR(p) model be $y(t) = \phi_1y(t-1) + \phi_2y(t-2)+\ldots+\phi_py(t-p) + \eta(t)$

where $\eta$ is the stationary excitation input of white Bernoulli distribution and the process are stationary.

Let the output of the process be corrupted with measurement noise $v(t)$ which is white Gaussian noise of zero -mean and unknown variance $\sigma^2_V$: $z_{ar}(t) = y(t) + v(t)$

$v(t)$ is uncorrelated with $\eta(t)$.

Problem 1: What will be the joint pdf for the noiseless and the noisy model?

Problem 2: In the application of blind system identification, do we assume that the excitation input $\eta$ is zero mean with unknown variance?

Can somebody please provide references and guidelines to proceed to find the solutions? Thank you.

Answer 1

Let $\Phi=(\phi_1,...,\phi_p)$ and $\mathbf{y}_{t-1} = (y_{t-1},...,y_{t-p})$

So your equations become $y_t = \Phi^{T}\mathbf{y}_{t-1} + \eta(t)$ and $z_{t} = y_{t} + v(t)$

thus $$ z_t - \Phi^{T}\mathbf{y}_{t-1} = \eta(t) + v(t) $$ Where $p$ is the parameter of the Bernoulli distributed variable $\eta$; note that $$E[\eta(t) + v(t)]=p$$ and $$var[\eta(t) + v(t)]=p(1-p)+\sigma^2$$ Where $\sigma^2$ is the variance of $v(t)$.

Using this we construct the likelihood function $$ \prod^{N}_{n=p+1} \mathcal{H}(z_t - \Phi^{T}\mathbf{y}_{t-1}\,|\,p\,,\,p(1-p)+\sigma^2) $$

where $\mathcal{H}$ is a pdf of unknown mean $p$ and variance $p(1-p)+\sigma^2$.

Personally I would assume $\mathcal{H}$ is a normal distribution since on average $\eta(t) + v(t) \sim \mathcal{N}(\,p\,,\,p(1-p)+\sigma^2)$

but this is fuzzy logic since it violates the assumption that $(\eta(t) + v(t))$ are identically distributed...but that's my 2 cents anyway.