# Unable to calculate the density function for AR

The model is an AR(p) process excited by a white Gaussian noise $\epsilon_t$, \begin{align} Y_t = &c+ \phi_1Y_{t-1} + \phi_2 Y_{t-2}+ \ldots+ \phi_p Y_{t-p} + \epsilon_t\\ \epsilon_t = &\mathcal{N}(0,\sigma^2)\\ \theta = &(c,\phi_1,\phi_2,\ldots,\phi_p,\sigma^2) \end{align} We collect first $p$ observations in the sample $(Y_1,Y_2,\ldots,Y_p)$ in a $p\times1$ vector $y_p$ which has mean vector $\mu_p$ with each element $\mu = \frac{c}{1-\phi_1-\phi_2 - \ldots - \phi_p}$ and $\sigma^2\mathbf{V}_p$ is the variance-covariance matrix

Question1: How to calculate the density function?

This is what I did, but I have doubt which is in the third term of the density function of the first $p$ observations, won't there be the power $p/2$ on $|V_p|$ instead of the correct power $1/2$?

The density of the first $p$ observations which I am getting is $f_{Y_p,Y_{p-1},\ldots,Y_1}(y_p,y_{p-1},\dots,y_1;\theta) = {(\frac{1}{\sqrt{(2 \pi \sigma^2 V_p)}})}^p\exp{(-\frac{(Y_p - \mu_p)'V_p^{-1}(Y_p- \mu_p)}{2 \sigma^2})}$

$= {(2 \pi)}^{-p/2} {(\sigma^2)}^{-p/2}{( |V_p^{-1}|)}^{p/2} \exp {(.)}$

which is incorrect according to the observations mentioned in my other Question https://dsp.stackexchange.com/questions/19182/unable-to-derive-crb-for-ar-model

Question2: What will be the complete density function and the likelihood?

• Avoid cross-posting on SE. Please read this question about cross-posting. – Gilles Nov 18 '14 at 21:40
• @Gilles:Will delete from here – Ria George Nov 19 '14 at 0:46

Question1: How to calculate the density function?

\begin{align} f_{Y_p,Y_{p-1},\ldots,Y_1}(y_p,y_{p-1},\dots,y_1;\mathbf{\theta}) = &\small (2\pi)^{-p/2} \left|\sigma^{-2} \mathbf{V}_p^{-1}\right|^{1/2}\exp{\left[-\frac{(\mathbf{y}_p - \mathbf{\mu}_p)'\mathbf{V}_p^{-1}(\mathbf{y}_p- \mathbf{\mu}_p)}{2 \sigma^2}\right]}\\ =&\small (2\pi)^{-p/2} (\sigma^{-2})^{p/2}\left| \mathbf{V}_p^{-1}\right|^{1/2}\exp{\left[-\frac{(\mathbf{y}_p - \mathbf{\mu}_p)'\mathbf{V}_p^{-1}(\mathbf{y}_p- \mathbf{\mu}_p)}{2 \sigma^2}\right]} \end{align}

As I said in my comments on this DSP question, you have an extra term with $\left| V_p^{-1}\right|$ both in your likelihood and loglikelihood.

Question2: What will be the complete density function and the likelihood?

The complete likelihood function is: \begin{align} f_{Y_T,Y_{T-1},\ldots,Y_1}(y_T,y_{T-1},\dots,y_1;\mathbf{\theta}) = & f_{Y_p,Y_{p-1},\ldots,Y_1}(y_p,y_{p-1},\dots,y_1;\mathbf{\theta}) \\ & \times \prod_{t=p+1}^T f_{Y_t|Y_{t-1},\ldots,Y_{t-p}}(y_t|y_{t-1},\dots,y_{t-p};\mathbf{\theta}) \end{align}

And the loglikelihood:

\begin{align} {\large\mathcal{L}}(\theta) = &\log f_{Y_T,Y_{T-1},\ldots,Y_1}(y_T,y_{T-1},\dots,y_1;\mathbf{\theta})\\ = &\log f_{Y_p,Y_{p-1},\ldots,Y_1}(y_p,y_{p-1},\dots,y_1;\mathbf{\theta}) +\log \prod_{t=p+1}^T f_{Y_t|Y_{t-1},\ldots,Y_{t-p}}(y_t|y_{t-1},\dots,y_{t-p};\mathbf{\theta})\\ =&-\frac{p}{2}\log(2\pi)-\frac{p}{2}\log(\sigma^2)+\frac{1}{2}\log\left| V_p^{-1}\right|-\frac{1}{2\sigma^2}(\mathbf{y}_p - \mathbf{\mu}_p)'\mathbf{V}_p^{-1}(\mathbf{y}_p- \mathbf{\mu}_p)\\ &-\frac{T-p}{2}\log(2\pi)-\frac{T-p}{2}\log(\sigma^2)\\ &-\frac{1}{2\sigma^2}\displaystyle\sum_{t=p+1}^T\left(y_t - c - \phi_1 y_{t-1}- \phi_2 y_{t-2}-\cdots- \phi_p y_{t-p}\right)^2\end{align}

• Thank you for your answer. But you forgot to mention for Answer1 why there will be no power $p/2$ on the variance matrix $|V_p|$. I think this is related to fundamentals which I am not aware of due to which I have mistakenly put the term $p/2$. Could you please shed some light on this? Lastly, I have a realted Question stats.stackexchange.com/questions/124549/… and struggling again to find the density and likelihood. Could you also have a look at this? – Ria George Nov 19 '14 at 3:03
• There is no $p/2$ on it because of the rules of calculating the determinant. If $a$ is a constant and $\mathbf{X}$ a $p \times p$ matrix, then $|a \mathbf{X}| = a^p |\mathbf{X}|$. – Gilles Nov 19 '14 at 3:20