The model is an AR(p) process excited by a white Gaussian noise $\epsilon_t$, \begin{align} Y_t = &c+ \phi_1Y_{t-1} + \phi_2 Y_{t-2}+ \ldots+ \phi_p Y_{t-p} + \epsilon_t\\ \epsilon_t = &\mathcal{N}(0,\sigma^2)\\ \theta = &(c,\phi_1,\phi_2,\ldots,\phi_p,\sigma^2) \end{align} We collect first $p$ observations in the sample $(Y_1,Y_2,\ldots,Y_p)$ in a $p\times1$ vector $y_p$ which has mean vector $\mu_p$ with each element $\mu = \frac{c}{1-\phi_1-\phi_2 - \ldots - \phi_p}$ and $\sigma^2\mathbf{V}_p$ is the variance-covariance matrix

Question1: How to calculate the density function?

This is what I did, but I have doubt which is in the third term of the density function of the first $p$ observations, won't there be the power $p/2$ on $|V_p|$ instead of the correct power $1/2$?

The density of the first $p$ observations which I am getting is $f_{Y_p,Y_{p-1},\ldots,Y_1}(y_p,y_{p-1},\dots,y_1;\theta) = {(\frac{1}{\sqrt{(2 \pi \sigma^2 V_p)}})}^p\exp{(-\frac{(Y_p - \mu_p)'V_p^{-1}(Y_p- \mu_p)}{2 \sigma^2})}$

$= {(2 \pi)}^{-p/2} {(\sigma^2)}^{-p/2}{( |V_p^{-1}|)}^{p/2} \exp {(.)}$

which is incorrect according to the observations mentioned in my other Question https://dsp.stackexchange.com/questions/19182/unable-to-derive-crb-for-ar-model

Question2: What will be the complete density function and the likelihood?

  • $\begingroup$ Avoid cross-posting on SE. Please read this question about cross-posting. $\endgroup$ – Gilles Nov 18 '14 at 21:40
  • $\begingroup$ @Gilles:Will delete from here $\endgroup$ – Ria George Nov 19 '14 at 0:46

Question1: How to calculate the density function?

\begin{align} f_{Y_p,Y_{p-1},\ldots,Y_1}(y_p,y_{p-1},\dots,y_1;\mathbf{\theta}) = &\small (2\pi)^{-p/2} \left|\sigma^{-2} \mathbf{V}_p^{-1}\right|^{1/2}\exp{\left[-\frac{(\mathbf{y}_p - \mathbf{\mu}_p)'\mathbf{V}_p^{-1}(\mathbf{y}_p- \mathbf{\mu}_p)}{2 \sigma^2}\right]}\\ =&\small (2\pi)^{-p/2} (\sigma^{-2})^{p/2}\left| \mathbf{V}_p^{-1}\right|^{1/2}\exp{\left[-\frac{(\mathbf{y}_p - \mathbf{\mu}_p)'\mathbf{V}_p^{-1}(\mathbf{y}_p- \mathbf{\mu}_p)}{2 \sigma^2}\right]} \end{align}

As I said in my comments on this DSP question, you have an extra term with $\left| V_p^{-1}\right|$ both in your likelihood and loglikelihood.

Question2: What will be the complete density function and the likelihood?

The complete likelihood function is: \begin{align} f_{Y_T,Y_{T-1},\ldots,Y_1}(y_T,y_{T-1},\dots,y_1;\mathbf{\theta}) = & f_{Y_p,Y_{p-1},\ldots,Y_1}(y_p,y_{p-1},\dots,y_1;\mathbf{\theta}) \\ & \times \prod_{t=p+1}^T f_{Y_t|Y_{t-1},\ldots,Y_{t-p}}(y_t|y_{t-1},\dots,y_{t-p};\mathbf{\theta}) \end{align}

And the loglikelihood:

\begin{align} {\large\mathcal{L}}(\theta) = &\log f_{Y_T,Y_{T-1},\ldots,Y_1}(y_T,y_{T-1},\dots,y_1;\mathbf{\theta})\\ = &\log f_{Y_p,Y_{p-1},\ldots,Y_1}(y_p,y_{p-1},\dots,y_1;\mathbf{\theta}) +\log \prod_{t=p+1}^T f_{Y_t|Y_{t-1},\ldots,Y_{t-p}}(y_t|y_{t-1},\dots,y_{t-p};\mathbf{\theta})\\ =&-\frac{p}{2}\log(2\pi)-\frac{p}{2}\log(\sigma^2)+\frac{1}{2}\log\left| V_p^{-1}\right|-\frac{1}{2\sigma^2}(\mathbf{y}_p - \mathbf{\mu}_p)'\mathbf{V}_p^{-1}(\mathbf{y}_p- \mathbf{\mu}_p)\\ &-\frac{T-p}{2}\log(2\pi)-\frac{T-p}{2}\log(\sigma^2)\\ &-\frac{1}{2\sigma^2}\displaystyle\sum_{t=p+1}^T\left(y_t - c - \phi_1 y_{t-1}- \phi_2 y_{t-2}-\cdots- \phi_p y_{t-p}\right)^2\end{align}

  • $\begingroup$ Thank you for your answer. But you forgot to mention for Answer1 why there will be no power $p/2$ on the variance matrix $|V_p|$. I think this is related to fundamentals which I am not aware of due to which I have mistakenly put the term $p/2$. Could you please shed some light on this? Lastly, I have a realted Question stats.stackexchange.com/questions/124549/… and struggling again to find the density and likelihood. Could you also have a look at this? $\endgroup$ – Ria George Nov 19 '14 at 3:03
  • 1
    $\begingroup$ There is no $p/2$ on it because of the rules of calculating the determinant. If $a$ is a constant and $\mathbf{X}$ a $p \times p$ matrix, then $|a \mathbf{X}| = a^p |\mathbf{X}|$. $\endgroup$ – Gilles Nov 19 '14 at 3:20

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.