# How to estimate maximum liklihood of a custom log likelihood function?

I am not very familiar with maximum likelihood estimation.

But I would like to test the null hypothesis $$\mu = 0, \sigma = 1, \rho = 0$$ by estimating the following model: $$z_t - \mu = \rho(z_{t-1} - \mu) + \epsilon_t$$

The log-likelihood function is $$-\frac{1}{2} log(2\pi) -\frac{1}{2} log(\frac{\sigma^2}{(1-\rho^2)}) - \frac{(z_1 - \mu/(1-\rho))^2}{2\sigma^2/(1-\rho^2)} \\ - \frac{T-1}{2} log(2\pi) - \frac{T-1}{2} log(\sigma^2) - \sum^T_{i=2} \frac{(z_t - \mu - \rho z_{t-1} )^2}{2\sigma^2}$$ , where $$\sigma^2$$ is the variance of $$\epsilon_t$$

The likelihood ratio statistic is $$LR = -2(L(0,1,0) - L(\hat\mu, \hat\sigma, \hat\rho))$$

Under the null hypothesis, the test statistic is distributed $$\chi^2(3)$$.

Where do I begin and how do I best estimate it in Matlab? Thanks in advance!

• I think the numerator in the third term in your log likelihood should read $(z_1 - \mu)^2$ and the numerator inside the sum should read something like $(z_t-\mu -\rho(z_{t-1}-\mu))^2$. May 23, 2020 at 16:07
• I cheked again and it should be correct. I got this from "Berkowitz: Testing Density Forecasts, With Applications to Risk Management" and also found it in Hamilton: Time series analysis (equation (5.2.9)). May 23, 2020 at 16:16
• No, I think either you or your sources mix up different parameterizations of the AR(1) model. The likelihood of the AR(1) model can be decomposed as $f(z_1)f(z_2|z_1)\cdots f(z_n|z_{n-1})$. Given your first equation, $z_1$ is normal with unconditional mean $\mu$ (which leads to the term $(z_1-\mu)^2$ in the log-likelihood). Similar considerations of the conditional means of subsequent $z_t$'s lead to the other expression in my first comment. May 23, 2020 at 16:31
• If this is self-study you should add the tag. May 23, 2020 at 16:37
• Thanks for your help and recommendations! Sry, I am new here on stats.stackexchange, but I will add the tag! May 23, 2020 at 16:56

The Maximum-Likelihood-Mathod assumes that you have a random variable $$X$$ and a probability density function $$f(x;\theta)$$ which is parametrized by $$\theta$$ (in general there is more than one parameter, e.g. $$\theta = (\mu, \sigma)$$ for the univariate gaussian distribution).
If you have $$N$$ samples from your random variable $$X$$, you can assume they are i.i.d (independent & identically distributed). Now you can calculate the likelihood of your observation as function of $$\theta$$

$$\mathcal{L}(\theta) = \prod_i^N f_{\theta}(x_i)$$ or as log-likelihood $$l(\theta) = \sum_i^N\text{log}\ f_{\theta}(x_i)$$.

Now the maximum likelihood methods tries to find the parameter(s) $$\theta$$ that maximize the likelihood function $$\mathcal{L}$$ resp. the log-likelihood function $$l$$,

$$\hat{\theta} = \underset{\theta}{\text{arg max}}\ \mathcal{L}(\theta)$$ or $$\hat{\theta} = \underset{\theta}{\text{arg max}}\ \mathcal{l}(\theta)$$.

This is usually done b setting the derivative of the likelihood function with respect to $$\theta$$ to zero, $$\frac{\partial \mathcal{L}}{\partial \theta} = 0$$, and solving for $$\theta$$. Since you already have a log-likelihood function, you can pick up from there to find $$\hat{\theta}$$.

Edit: To make sure you have a maximum and not a minimum, also check that the second derivative satisfies $$\frac{\partial^2\mathcal{L}}{\partial \theta^2} < 0$$.

• There also is the issue of showing the the values you find are indeed minima or maxima, perhaps by a second-derivative test (Hessian matrix eigenvalues).
– Dave
May 23, 2020 at 15:39
• Thanks a lot for your answers! As @Tinu mentioned, I already have my log-likelihood function. Is this the log of the densities? And how do I proceed from there? Because I would have to take the derivative w.r.t. the parameters, right? How is this best done in a software (preferably Matlab)? May 23, 2020 at 16:07
• This will not in this case. You need to maximise the likelihood numerically. May 23, 2020 at 16:36
• If you have samples $x_1, ..., x_N$ you can use numerical optimization techniques from matlabs optimization toolbox. If you like to have an analytic expression (which is preferable i.m.o.) you can try maple or mathematica which are able to do symbolic calculations or by pen & paper ;)
– Tinu
May 23, 2020 at 16:37
• @Tinu Thanks, I think I prefer the matlab optimization over pen & paper ;) So I would have to optimize the sum of the log-likelihood function, right? Can this be done by the "fminsearch" command? So minimizing the negative sum of log-likelihood functions? May 23, 2020 at 16:54