Proving that a random walk using a maximum likelihood estimator can diverge to infinity Consider a sequence of continuous random variables $\{X_n\}^\infty_{n=1}$ that are independent and identically distributed under the probability density function $f_\theta (x)$, where $\theta \in [\ell ,u]$ is an unknown but deterministic parameter, $\ell < 0 <u$ and $\theta \neq 0$.Consider the random sequence:
\begin{equation}
S_n = \sum_{i=2}^n \log\frac{f_{\hat{\theta}_{1:i-1}}(X_i)}{f_0(X_i)}
\end{equation}
where $\hat{\theta}_{1:i-1}$ is the maximum likelihood estimate of the parameter $\theta$ using the observations $\{X_n\}^{i-1}_{n=1}$, and $S_0 \triangleq S_1 \triangleq 0$. Let $ \tau =\inf\{n \geq 2 : S_n \leq 0\}$. I am trying to show that $S_n$ may diverge to infinity before becoming non-positive, with non-zero probability, i.e., 
\begin{equation}
\mathbb{P}(\tau = \infty) >0.
\end{equation}
My intuition says that this should hold due to the fact that $\hat{\theta}_{1:i-1}$ converges to $\theta$ almost surely as $i \rightarrow \infty$. Do you think this result should hold for the case that the maximum likelihood estimator is used to estimate the parameter $\theta$. I would really appreciate if anyone could provide any results that could be useful to show this claim! Thanks!
 A: The following is more of an extended comment with explicit calculations for the Gaussian case to illustrate the idea.
Consider the unit variance case so I can avoid typing $\sigma$, denote the parameter of interest (mean) by $\mu > 0$ for both laziness and aesthetics. Thus we are interested in
$$
f_\mu(x) = c\exp(-(x-\mu)^2/2)
$$
where $c$ is the corresponding normalising constant.
Continue to let $X_1,\ldots$ be your random sample from $f_\mu$.  The MLE for $\mu$ is just the sample mean,
$$
\hat \mu_n = \frac{1}{n}\sum_{i=1}^n X_i.
$$
Take any $\epsilon > 0$.  The SLLN implies 
$$
\mathbb{P}(| \hat \mu_n - \mu| < \epsilon, n > N) \to 0
$$
as $N\to \infty$.  So for $N$ large enough we can assume $| \hat \mu_n - \mu| < \epsilon$ for $n > N$ off an event with arbitrarily small probability.
Observe
$$
\ln f_m(x)/f_0(x) = \ln\exp(-(x-m)^2/2 + x^2/2) = m(2x-m)/2
$$
and 
$$
m(2x -m) > F(x) := (\mu \pm \epsilon)(2x - \mu - \epsilon)
$$
for $|m - \mu| < \epsilon$ (where the $\pm$ is depending on whether $m-2x \ge 0$ or not).  
So if $\epsilon < \mu$ then no matter the exact value of $m \in (\mu-\epsilon, \mu+\epsilon)$,
$$
\ln f_m(X_n)/f_0(X_n) > F(X_n)
$$
where the right hand side is a random variable with positive expectation.  
In particular, on an event of arbitrarily high probability, 
$$
\ln f_{\hat \mu_n}(X_{n+1})/f_0(X_{n+1}) > F(X_{n+1})
$$
for all $n > N$ (by choosing fixed $N$ large enough).
Now, the RW $\tilde S_k := \sum_{i=1}^k F(X_{N+i})$, $k \ge 0$ (starting from 0) has i.i.d increments with positive expectation so explodes to infinity before hitting zero with positive probability not depending on $N$.  So WLOG choose $N$ in the previous step large enough so that $| \hat \mu_n - \mu| < \epsilon$ for $n > N$ fails on an event with smaller probability.  Thus, the intersection of $(\tilde S_k)$ exploding before going negative and $| \hat \mu_n - \mu| < \epsilon$ for $n > N$ has positive probability.  But on the latter event $(S_{N+k} - S_N)$ is lower bounded by $\tilde S_k$ so also explodes before hitting zero.
Now, you just need to show that the probability that $S_1>0,S_2 > 0,\ldots, S_N > 0$ (finitely many variables) is positive (with $| \hat \mu_n - \mu| < \epsilon$ for $n > N$ still holding off an event with arbitrarily small probability), which seems straightforwards.
