MLE Derivation for AR Model So I am trying to derive the MLE for an AR(1) model. Here are my thoughts thus far:
The AR process is: $z_t = \delta + \psi_1z_{t-1} + \epsilon_t$
The expected value of $z_t = \frac{\delta}{1 - \psi_1}$.
The variance of $z_t = \frac{1}{1 - \psi_1^2}$.
So this is where I am getting caught up.
I have the PDF of $z_t$ as:
\begin{align}
f(z_t;\theta) &= (2 \pi \sigma^2)^{-\frac{1}{2}} 
    \exp \left [-\frac{1}{2} \left (\frac{z_t - \mathbb{E}[z_t]}{\sqrt{\sigma^2}}\right )  \right] \\
    &= \left (2 \pi \frac{1}{1 - \psi_1^2} \right )^{-\frac{1}{2}}
    \exp \left [-\frac{1}{2} \left (\frac{z_t - \frac{\delta}{1 - \psi_1}}
    {\sqrt{\frac{1}{1 - \psi_1^2}}}  \right )^2  \right] \\
    &= \left (2 \pi \frac{1}{1 - \psi_1^2} \right )^{-\frac{1}{2}}
    \exp \left [-\frac{1}{2} \left (\frac{ \left(z_t - \frac{\delta}{1 - \psi_1} \right )^2}{\frac{1}{1 - \psi_1^2}}  \right )  \right] \\
    &= \left (2 \pi \frac{1}{1 - \psi_1^2} \right )^{-\frac{1}{2}}
    \exp \left [-\frac{1 - \psi_1^2}{2} \left( z_t - \frac{\delta}{1 - \psi_1} \right)^2  \right]
\end{align}
Now, can I assume i.i.d. here? I feel like no because then I would have a time series that is just white noise right? However, if I did assume i.i.d., I would have:
$\mathscr{L} = \prod_{t=1}^T \left (2 \pi \frac{1}{1 - \psi_1^2} \right )^{-\frac{1}{2}}
    \exp \left [-\frac{1 - \psi_1^2}{2} \left( z_t - \frac{\delta}{1 - \psi_1} \right)^2  \right]$
And then from here what exactly would my log likelihood function be? I feel like I am totally screwing this up but this is what I have for it:
$\ln \mathscr{L} = -\frac{T}{2} \ln \left ( 2 \pi \frac{1}{1 - \psi_1^2} \right )
- \frac{(1 - \psi_1^2) \sum_{t=1}^T \left (z_t - \frac{\delta}{1 - \psi_1}  \right )^2}{2}$
Any help is greatly appreciated! Thank you!!
 A: No, the $z_t$ are not independent, so that doesn't work.
The likelihood is a joint probability, so start there rather than from a single observation:
$$\mathcal{L}(\theta) = p(z_1, ..., z_T| \theta)$$
The recursive definition of the process (the first equation you show) gives you the transition distribution directly:
$$z_t | (z_1,...,z_{t-1}, \theta) \sim \mathcal{N}(\delta + \psi_1 z_{t-1}, \sigma^2)$$
So, it would be ideal if we could express our joint probability in terms of these transition distributions, then we can just plug them in. You can recursively condition on the previous observations to decompose the joint distribution like this:
$$\begin{align}
\mathcal{L}(\theta) &= p(z_1, ..., z_T| \theta) \\
&=  p(z_T|z_1,...,z_{T-1};\theta) \cdot  p(z_1,...,z_{T-1}|\theta)\\
&=  p(z_T|z_1,...,z_{T-1};\theta) \cdot  p(z_{T-1}|z_1,...,z_{T-2};\theta) \cdot p(z_1,...,z_{T-2}|\theta)\\
&= \quad... \\
&= p(z_T|z_1,...,z_{T-1};\theta) \cdot p(z_{T-1}|z_1,...,z_{T-2};\theta) \cdot ... \cdot p(z_1|\theta)
\end{align}$$
Plug in the normal transition distributions from above and you're (almost) done. There's a small issue: what is $p(z_1|\theta)$? Many definitions of AR(1) processes do not actually make this explicit. Typically, what is implied is that it has the "stationary distribution":
$$z_1 | \theta \sim \mathcal{N}\left(\frac{\delta}{1-\psi_1},\frac{\sigma^2}{1-\psi_1^2}\right)$$
A: I'm not directly answering your question, but a quick note on the construction of the likelihood function in your model. The likelihood of a $T$-sized sample of $\mathbf{e} = \left[ \epsilon_{1}, \, \epsilon_{2}, \ldots, \, \epsilon_{T} \right]^{\mathsf{T}}$ of i.i.d. normal distributed $\epsilon \sim N(0,\sigma^{2})$ is
$$
L(\mathbf{e}) = (2 \pi \sigma^{2})^{\frac{T}{2}} \exp \left[ \frac{- \mathbf{e}^\mathsf{T} \mathbf{e}}{2\sigma^{2}}   \right]
$$
But obviously you observe $\mathbf{z} = \left[ z_{1}, \, z_{2}, \ldots, \, z_{T} \right]^{\mathsf{T}}$ instead of $\mathbf{e}$. From your AR(1) setup you have $\mathbf{e} = \mathbf{G} \mathbf{z} - \delta \mathbf{I}_{T}$ where
$$
\mathbf{G} = \begin{bmatrix}
\sqrt{1-\psi_{1}^2} & 0 & 0 & \cdots & 0 \\
-\psi_{1} & 1 & 0 & \cdots & 0 \\
0 & -\psi_{1} & 1 & \cdots &  0 \\
\vdots & \vdots & \vdots & \ddots & \vdots \\
0 & 0 & 0 & \cdots & 1
\end{bmatrix},
$$
(see Prais & Winsten, eqn. 15). Then
$$L(\mathbf{z}) = L(\mathbf{e}) \left| \frac{d \mathbf{e}}{d\mathbf{z}} \right|$$
where $\left| \frac{d \mathbf{e}}{d\mathbf{z}} \right|$ is the Jacobian (determiant) of the transformation, in this case $\operatorname{det} \mathbf{G}$, which works out to be $\left| 1 -\psi_{1}^{2} \right|^{\frac{1}{2}}$ because of all the off-diagonal zeros in $\mathbf{G}$. See Beach & MacKinnon (1978) for more details.
