# Proving that a random walk using a maximum likelihood estimator can diverge to infinity

Consider a sequence of continuous random variables $\{X_n\}^\infty_{n=1}$ that are independent and identically distributed under the probability density function $f_\theta (x)$, where $\theta \in [\ell ,u]$ is an unknown but deterministic parameter, $\ell < 0 <u$ and $\theta \neq 0$.Consider the random sequence:

$$S_n = \sum_{i=2}^n \log\frac{f_{\hat{\theta}_{1:i-1}}(X_i)}{f_0(X_i)}$$

where $\hat{\theta}_{1:i-1}$ is the maximum likelihood estimate of the parameter $\theta$ using the observations $\{X_n\}^{i-1}_{n=1}$, and $S_0 \triangleq S_1 \triangleq 0$. Let $\tau =\inf\{n \geq 2 : S_n \leq 0\}$. I am trying to show that $S_n$ may diverge to infinity before becoming non-positive, with non-zero probability, i.e., $$\mathbb{P}(\tau = \infty) >0.$$ My intuition says that this should hold due to the fact that $\hat{\theta}_{1:i-1}$ converges to $\theta$ almost surely as $i \rightarrow \infty$. Do you think this result should hold for the case that the maximum likelihood estimator is used to estimate the parameter $\theta$. I would really appreciate if anyone could provide any results that could be useful to show this claim! Thanks!

• Agreed with your intuition: for large $n$ we have $S_{n+1} \approx S_n + \xi_{n+1}$ where $\xi_n = \frac{\ln f_\theta(X_{n}) }{f_{0}(X_{n})}$ is independent of everything preceding in the process with $\mathbb{E}\xi_n > 0$. So it is eventually like a random walk with positive drift. However $\theta = 0$ is more problematic..... in fact I can't imagine anything other than $\mathbb{P}(\tau = \infty) = 0$ for that case – P.Windridge May 22 '18 at 21:39
• It is assumed that $\theta \neq 0$ for this problem. I am not sure if any stronger assumptions should be made. The analysis here seem more difficult since we have non i.i.d. and a lot of tools can not be used. However I was wondering whether the claim could be shown for the case that $f_\theta$ belongs to an exponential family, or is Gaussian. – SpawnKilleR May 22 '18 at 22:01
• Oh I missed the $\theta \neq 0$ assumption. As for other assumptions it would be helpful to assume you have enough regularity for the MLE to converge. Anyway, in the argument I was outlining you approximately lower bound $S_n$ by i.i.d increments for large $n$, say $n > N$. Then you combine that with a separate argument to say $S_N > 0$ with positive probability. – P.Windridge May 22 '18 at 22:43
• I'll try to explain in more detail. Choose $\epsilon > 0$ small enough such that $\mathbb{E}\inf F(\theta';X_n) > 0$ where the inf is taken over $\theta' \in [\theta-\epsilon,\theta + \epsilon]$ and $F(\theta',x) = \ln f_{\theta'}(x) / \ln f_0(x)$. You have $\hat \theta_n \to \theta$, so eventually $\hat\theta_n \in [\theta-\epsilon,\theta+\epsilon]$ and your increments are bounded below by i.i.d variables. – P.Windridge May 22 '18 at 22:47
• I'm not sure I understand what you mean by " bounded below by i.i.d variables". Which are those random variables and why are they a bound for the increments? Also, isn't that statement you are talking about asymptotic? I can not see the connection with the statement to be proven. Thank you so much for your time! I really appreciate the help! – SpawnKilleR May 23 '18 at 0:37

The following is more of an extended comment with explicit calculations for the Gaussian case to illustrate the idea.

Consider the unit variance case so I can avoid typing $\sigma$, denote the parameter of interest (mean) by $\mu > 0$ for both laziness and aesthetics. Thus we are interested in $$f_\mu(x) = c\exp(-(x-\mu)^2/2)$$ where $c$ is the corresponding normalising constant.

Continue to let $X_1,\ldots$ be your random sample from $f_\mu$. The MLE for $\mu$ is just the sample mean, $$\hat \mu_n = \frac{1}{n}\sum_{i=1}^n X_i.$$

Take any $\epsilon > 0$. The SLLN implies $$\mathbb{P}(| \hat \mu_n - \mu| < \epsilon, n > N) \to 0$$ as $N\to \infty$. So for $N$ large enough we can assume $| \hat \mu_n - \mu| < \epsilon$ for $n > N$ off an event with arbitrarily small probability.

Observe $$\ln f_m(x)/f_0(x) = \ln\exp(-(x-m)^2/2 + x^2/2) = m(2x-m)/2$$

and $$m(2x -m) > F(x) := (\mu \pm \epsilon)(2x - \mu - \epsilon)$$ for $|m - \mu| < \epsilon$ (where the $\pm$ is depending on whether $m-2x \ge 0$ or not).

So if $\epsilon < \mu$ then no matter the exact value of $m \in (\mu-\epsilon, \mu+\epsilon)$, $$\ln f_m(X_n)/f_0(X_n) > F(X_n)$$ where the right hand side is a random variable with positive expectation.

In particular, on an event of arbitrarily high probability, $$\ln f_{\hat \mu_n}(X_{n+1})/f_0(X_{n+1}) > F(X_{n+1})$$ for all $n > N$ (by choosing fixed $N$ large enough).

Now, the RW $\tilde S_k := \sum_{i=1}^k F(X_{N+i})$, $k \ge 0$ (starting from 0) has i.i.d increments with positive expectation so explodes to infinity before hitting zero with positive probability not depending on $N$. So WLOG choose $N$ in the previous step large enough so that $| \hat \mu_n - \mu| < \epsilon$ for $n > N$ fails on an event with smaller probability. Thus, the intersection of $(\tilde S_k)$ exploding before going negative and $| \hat \mu_n - \mu| < \epsilon$ for $n > N$ has positive probability. But on the latter event $(S_{N+k} - S_N)$ is lower bounded by $\tilde S_k$ so also explodes before hitting zero.

Now, you just need to show that the probability that $S_1>0,S_2 > 0,\ldots, S_N > 0$ (finitely many variables) is positive (with $| \hat \mu_n - \mu| < \epsilon$ for $n > N$ still holding off an event with arbitrarily small probability), which seems straightforwards.

• Thanks for the answer! I will try to see if a similar proof works for the exponential family case. – SpawnKilleR May 25 '18 at 21:34
• So as we said before there is a problem when $\mu =0$. In the proof you also have to be able to set your $\epsilon$ to be smaller than $\mu$. Since $\mu$ can go arbitrary close to $0$, doesn't that mean that for those cases $\epsilon$ will be very close to $0$. Will that lead to $N$ becoming unbounded, and if yes wouldn't that be a problem when proving the last simple argument? – SpawnKilleR May 31 '18 at 6:24
• Yep the choice of $N$ and $\epsilon$ depends on $\mu$ but the latter is assumed constant, i.e. for the problem stated the argument works but indeed it's not uniform in $\mu$ might cause problems in practice? Btw I no longer think the "last simple part" is quite as easy as I initially thought, or rather it is but then you need to do the main part conditioned on $S_1,\ldots,S_n$ being positive. – P.Windridge May 31 '18 at 11:05