2
$\begingroup$

The model is an AR(p) process excited by a white Gaussian noise $\epsilon_t$, \begin{align} Y_t = &c+ \phi_1Y_{t-1} + \phi_2 Y_{t-2}+ \ldots+ \phi_p Y_{t-p} + \epsilon_t\\ \epsilon_t = &\mathcal{N}(0,\sigma^2)\\ \theta = &(c,\phi_1,\phi_2,\ldots,\phi_p,\sigma^2) \end{align} We collect first $p$ observations in the sample $(Y_1,Y_2,\ldots,Y_p)$ in a $p\times1$ vector $y_p$ which has mean vector $\mu_p$ with each element $\mu = \frac{c}{1-\phi_1-\phi_2 - \ldots - \phi_p}$ and $\sigma^2\mathbf{V}_p$ is the variance-covariance matrix

Question1: How to calculate the density function?

This is what I did, but I have doubt which is in the third term of the density function of the first $p$ observations, won't there be the power $p/2$ on $|V_p|$ instead of the correct power $1/2$?

The density of the first $p$ observations which I am getting is $f_{Y_p,Y_{p-1},\ldots,Y_1}(y_p,y_{p-1},\dots,y_1;\theta) = {(\frac{1}{\sqrt{(2 \pi \sigma^2 V_p)}})}^p\exp{(-\frac{(Y_p - \mu_p)'V_p^{-1}(Y_p- \mu_p)}{2 \sigma^2})}$

$= {(2 \pi)}^{-p/2} {(\sigma^2)}^{-p/2}{( |V_p^{-1}|)}^{p/2} \exp {(.)}$

which is incorrect according to the observations mentioned in my other Question https://dsp.stackexchange.com/questions/19182/unable-to-derive-crb-for-ar-model

Question2: What will be the complete density function and the likelihood?

$\endgroup$
2
  • $\begingroup$ Avoid cross-posting on SE. Please read this question about cross-posting. $\endgroup$
    – Gilles
    Nov 18, 2014 at 21:40
  • $\begingroup$ @Gilles:Will delete from here $\endgroup$
    – Ria George
    Nov 19, 2014 at 0:46

1 Answer 1

1
$\begingroup$

Question1: How to calculate the density function?

\begin{align} f_{Y_p,Y_{p-1},\ldots,Y_1}(y_p,y_{p-1},\dots,y_1;\mathbf{\theta}) = &\small (2\pi)^{-p/2} \left|\sigma^{-2} \mathbf{V}_p^{-1}\right|^{1/2}\exp{\left[-\frac{(\mathbf{y}_p - \mathbf{\mu}_p)'\mathbf{V}_p^{-1}(\mathbf{y}_p- \mathbf{\mu}_p)}{2 \sigma^2}\right]}\\ =&\small (2\pi)^{-p/2} (\sigma^{-2})^{p/2}\left| \mathbf{V}_p^{-1}\right|^{1/2}\exp{\left[-\frac{(\mathbf{y}_p - \mathbf{\mu}_p)'\mathbf{V}_p^{-1}(\mathbf{y}_p- \mathbf{\mu}_p)}{2 \sigma^2}\right]} \end{align}

As I said in my comments on this DSP question, you have an extra term with $\left| V_p^{-1}\right|$ both in your likelihood and loglikelihood.

Question2: What will be the complete density function and the likelihood?

The complete likelihood function is: \begin{align} f_{Y_T,Y_{T-1},\ldots,Y_1}(y_T,y_{T-1},\dots,y_1;\mathbf{\theta}) = & f_{Y_p,Y_{p-1},\ldots,Y_1}(y_p,y_{p-1},\dots,y_1;\mathbf{\theta}) \\ & \times \prod_{t=p+1}^T f_{Y_t|Y_{t-1},\ldots,Y_{t-p}}(y_t|y_{t-1},\dots,y_{t-p};\mathbf{\theta}) \end{align}

And the loglikelihood:

\begin{align} {\large\mathcal{L}}(\theta) = &\log f_{Y_T,Y_{T-1},\ldots,Y_1}(y_T,y_{T-1},\dots,y_1;\mathbf{\theta})\\ = &\log f_{Y_p,Y_{p-1},\ldots,Y_1}(y_p,y_{p-1},\dots,y_1;\mathbf{\theta}) +\log \prod_{t=p+1}^T f_{Y_t|Y_{t-1},\ldots,Y_{t-p}}(y_t|y_{t-1},\dots,y_{t-p};\mathbf{\theta})\\ =&-\frac{p}{2}\log(2\pi)-\frac{p}{2}\log(\sigma^2)+\frac{1}{2}\log\left| V_p^{-1}\right|-\frac{1}{2\sigma^2}(\mathbf{y}_p - \mathbf{\mu}_p)'\mathbf{V}_p^{-1}(\mathbf{y}_p- \mathbf{\mu}_p)\\ &-\frac{T-p}{2}\log(2\pi)-\frac{T-p}{2}\log(\sigma^2)\\ &-\frac{1}{2\sigma^2}\displaystyle\sum_{t=p+1}^T\left(y_t - c - \phi_1 y_{t-1}- \phi_2 y_{t-2}-\cdots- \phi_p y_{t-p}\right)^2\end{align}

$\endgroup$
2
  • $\begingroup$ Thank you for your answer. But you forgot to mention for Answer1 why there will be no power $p/2$ on the variance matrix $|V_p|$. I think this is related to fundamentals which I am not aware of due to which I have mistakenly put the term $p/2$. Could you please shed some light on this? Lastly, I have a realted Question stats.stackexchange.com/questions/124549/… and struggling again to find the density and likelihood. Could you also have a look at this? $\endgroup$
    – Ria George
    Nov 19, 2014 at 3:03
  • 1
    $\begingroup$ There is no $p/2$ on it because of the rules of calculating the determinant. If $a$ is a constant and $\mathbf{X}$ a $p \times p$ matrix, then $|a \mathbf{X}| = a^p |\mathbf{X}|$. $\endgroup$
    – Gilles
    Nov 19, 2014 at 3:20

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.