1
$\begingroup$

I am using a direct likelihood to estimate the parameters of a discrete nonlinear state space model

\begin{equation} \begin{aligned} x_k & = f(x_{k-1},\theta)+q_k-1\\ y_k & = f(x_k,\theta)+r_k \end{aligned} \end{equation}

I used an UKF to calculate The log-likelihood expression:

$L(\theta) = \sum_{k=1}^T -\frac{1}{2}\log [(2\pi)^d\vert \Sigma\vert] - \frac{1}{2}(y_k-\mu)^T \Sigma^{-1} (y_k-\mu) $

where $\mu=\hat{y}_{k|k-1}$ and $\Sigma=P_{yy,k|k-1}$ and evaluated during the UKF filtering recursion and $d$ is the observation dimension.

I generated a synthetic data (Observation data $y_k$ needed by the UKF) based on my model with a vector of the $9$ true parameters $\theta = [0.6667 \hspace{0.2 cm} 1.5 \hspace{0.2 cm} 0.6667 \hspace{0.2 cm} 1.5 \hspace{0.2 cm} 2.5 \hspace{0.2 cm} 1.6 \hspace{0.2 cm} 0.5 \hspace{0.2 cm} 0.6 \hspace{0.2 cm} 10]$.

When I calculated the log-likelihood based on the true parameters gave me the numerical value $9,0841e+3$ (the software that I am using it is the MATLAB).

Now I tried to use the Maximum Likelihood Estimation MLE to estimate the parameters of the model from the synthetic data generated before.

I used the fmincon from MATLAB on the objective function which is the ($negative loglikelihood$ function because fmincon search for the minimum of the objective function, but what I want is the maximum, so minimize $-L(\theta)$ = maximize $L(\theta)$ ) to find the global minimum at the optimal point (the true parameters), and I used the (options = optimset('Display','iter') to display the value of the loglikelihood at each iteration.

When the optimization finished I get an optimal point far from the true parameters and the loglikelihood take the value $9,08485e+3$.

What I did not understand, is that the value of the loglikelihood $9,0841e+3$ at the true parameters is supposed to be the global maximum, but the optimization gave us the biggest value at different point $\theta = [0.93 \hspace{0.2 cm} 1.6347 \hspace{0.2 cm} 1 \hspace{0.2 cm} 1.5850 \hspace{0.2 cm} 2.5626 \hspace{0.2 cm} 1.1 \hspace{0.2 cm} 0.4904 \hspace{0.2 cm} 0.5991 \hspace{0.2 cm} 10.2040]$., so where is the problem.

Knowing that the UKF reconstructs the state space variables of the model very well.

$\endgroup$
1
$\begingroup$

FMINCON is a local optimizer. Even if it works as it is supposed to, it will find a local minimum (maximum for your problem), not necessarily the global minimum (maximum for your problem). You haven't shown us what your function $f$ is, but I am guessing that the log-Likelihood is not concave, i.e., that the negative log-likelihood which you are minimizing is convex. Therefore, there may be one or more local minima which is not globally optimal.

The difference in objective value between the FMINCON optimum and what you say is the true optimum is rather small, and one might think it could be a question of convergence tolerance; however, you claim the argument values are quite different (although you haven't shown us what they are), so it could just be that there are multiple local optima with similar objective values. it could even be that FMINCOM found another global minimum, and the small difference in optimal objective value is just a matter of convergence tolerance.

| cite | improve this answer | |
$\endgroup$
  • $\begingroup$ So I am dealing with some data whose likelihood seems to have several maxima. In this cas how do I estimate the parameters properly. (for note, I did some corrections to the question) $\endgroup$ – user137684 Dec 4 '16 at 23:52
  • $\begingroup$ It sounds like you have a model with at least 2 essentially equal likelihood optimal "points". You haven't shown your function $f$. Obviously what you are doing is just for practice or "fun", because you know the true solution. Does the function have (approximately) the same or different values for various values of $x_k$ when using one optimal parameter set vs. the other? $\endgroup$ – Mark L. Stone Dec 5 '16 at 0:00
  • $\begingroup$ $f$ is the log-likelihood function $L(\theta)$, this is a parameters estimation by Maximum Likelihood Estimation MLE when we approximate the log-likelihood using gaussian filtering. my model is a modelization of the neuronal activity of the human brain .I generated synthetic data with the true $theoretical$ values of parameters, to see if the fmincon is accurate, befor using this method on $Real \hspace{0.2cm} observation \hspace{0.2cm} data$ taken from a real experiment, the purpose here is the validation of the model parameters from the $Real \hspace{0.2cm} data$ . $\endgroup$ – user137684 Dec 5 '16 at 10:25
  • $\begingroup$ I don't know anything about your application area, so I can not comment on how well you have modeled anything or what the properties of your model should be. But maybe the parameters of your model are not identifiable - for example, estimate $a$ and $b$ in $f(x,a,b) = (a+b)x^2$. Any combination of $a$ and $b$ having the same sum would result in an equivalent model and a numerical optimizer, such as FMINCON, could return any combination achieving the optimal sum, $a+b$. $\endgroup$ – Mark L. Stone Dec 5 '16 at 13:02

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.