# Does the MLE-Kalman prediction maximize the likelyhood of the prediction?

The question is the following. Say I have observations of a Gaussian stochastic process ($\{x_i\}_{i=1}^n$) for which is convenient to use the state space formalism (and Kalman recursions) to describe it. Let its log-likelyhood function be $f(x;\theta)$, which depends on some parameter(s) $\theta$. In order calculate the log-likelyhood, we use the Kalman recursions.

Let $\hat{\theta}$ be the maximum likelyhood estimator of $\theta$. We use the Kalman prediction and $\hat{\theta}$ to get the prediction $\hat{x}_{n+1}$.

Le us consider now the following procedure. Let us assume $x_{n+1}$ and consider it as a parameter together with $\theta$. Then, we calculate the Kalman recursions but taking $x_{n+1}$ as given, that is, instead of using the innovations we use the smoothing solutions to compare with $x_1, x_2,...x_n$. Then we maximize likelyhood respect to $\theta$ and $x_{n+1}$, and we take the latter as the estimators of $\theta$ and $x_{n+1}$, respectively.

1. Is this a sensible procedure to calculate a likelyhood function with $x_{n+1}$ as a parameter?
2. In case it is, are these two ways of estimating $x_{n+1}$ and $\theta$ the same?
• After studying a bit more, it seems that under gaussian hypotheses the Kalman prediction actually gives you the conditional expectation of $x_{n+1}$ given the previous data. This is the best possible estimator (in min-squared sense) based on functions of the previous variables. Therefore, maximizing the likelyhood of the observed data $\{x_i\}_{i=1}^n$ given $x_{n+1}$ - which would be obtaining the MLE of $x_{n+1}$ - is necessarily worse or equally good as the Kalman prediction. – Enredanrestos Aug 7 '18 at 5:20

This is not a complete answer, but a short example which shows that: (i) the two approaches are not equivalent, (ii) the Kalman prediction gives a better solution. Let $x_1$ and $x_2$ observations a zero-centered, mean reverting Gaussian process. Say, $$x_1\sim N(0,\sigma^2)$$ $$x_{n+1}|x_{n} \sim N(a x_n,(1-a^2)\sigma^2),$$ with $0<a<1$ and $\sigma>0$ given. Then, the Kalman prediction of the second term based on one observation is $x_{2|1}=ax_1$. The PDF of $(x_1,x_2)$ is $$f(x_1,x_2)=\frac{1}{2\pi\sigma^2\sqrt{1-a^2}}\exp\left(-\frac{1}{2}\left(\frac{x_1^2}{\sigma^2}+\frac{(x_2-ax_1)^2}{(1-a^2)\sigma^2}\right)\right)~~.$$ This way of writing the density function hides a bit the fact that it is symmetrical in $x_1$ and $x_2$. In any case, it is not difficult to prove that \begin{align} \ln f(x_2|x_1)+\text{ct.}&\propto -(x_2-a x_1)^2\\ \ln f(x_1|x_2)+\text{ct.}&\propto -(x_1-a x_2)^2~~.\tag{1}\\ \end{align} Therefore $\mathbb{E}(x_2|x_1)=a x_1$ is the best estimator of $x_2$ based on $x_1$, and it corresponds to the Kalman prediction (side note: $\mathbb{E}(x_1|x_2)=a x_2$, which is rather surprising. I think that it has to do with this stochastic process being reversible).
However, as derived from $(1)$, treating $x_2$ as a parameter of the likelyhood function based on a single observation ($x_1$) and maximizing it, gives $\hat{x}_2=x_1/a$, which is certainly worse and unintuitive compared with the Kalman result. What is the main maximum-likelyhood hypothesis which is failing here? The MLE does not seem to get any better with more terms in the series.