# Estimate the second moment of a latent variable using a conditionally unbiased proxy

The Setup: Let $X_t$ denote an unobservable stochastic sequence where the first two unconditional moments are finite constants; ie $\mathbb{E} X_t = \mu < \infty$ and $\mathbb{E} X_t^2 = \gamma < \infty$. Assume that $X_t$ obeys some minimum set of regularity conditions such that a weak law of large numbers applies to its first two sample moments; ie conditions that limit the dependence in the sequence and bound the higher order moments.

Let $Y_t$ denote a conditionally unbiased proxy for $X_t$. That is, $\mathbb{E} [Y_t | X_t] = X_t$, and hence $\mathbb{E} Y_t = \mathbb{E} X_t = \mu$. Assume that $Y_t$ also obeys a minimum set of regularity conditions such that a weak law of large numbers applies to its first two sample moments.

Let $S_T = T^{-1} \sum_{t=1}^T Y_t$. Given the assumptions on $X_t$ and $Y_t$, it follows that $\mathbb{E} S_T = \mu$ and $S_T \overset{\mathbb{P}}{\rightarrow} \mu$, hence $S_T$ is a consistent, unbiased estimator for $\mu$

My Question: Given the setup, how can I estimate $\mathbb{E} X_t^2 = \gamma$?

Things I Know: From Slutsky's theorem, $(S_T)^2 \overset{\mathbb{P}}{\rightarrow} (\mathbb{E} X_t)^2 \neq \mathbb{E} X_t^2 = \gamma$, via Jensen's inequality, so in general $(S_T)^2$ will be a poor estimator for $\gamma$. Also, $T^{-1} \sum_{t=1}^T Y_t^2$ may not be a good estimator since the assumption set does not guarantee that $\mathbb{E} X_t^2 = \mathbb{E} Y_t^2$.

Final Note: I would be happy with any answer that improves upon the properties of $(S_T)^2$. So an approximation type estimator obtained via some sort of series expansion would be great. That's what I've mainly been looking into thus far, but with little luck. Even suggestions regarding expansions I might look into would be most welcome.

Thanks in advance to all responders.

• Hi, Colin. This question looks interesting, but I have several questions about it. Perhaps they are just notational issues or unstated assumptions, but much of the math in the question doesn't make complete sense. Are the means $\mu_t = \mathbb E X_t$ constant? (Writing something like $S_T \stackrel{p}{\to} \mathbb E X_t$ doesn't make sense, otherwise.) What other assumptions do you make, but, perhaps, haven't stated here? The more structure you can give to the problem, the better the potential answer. Feb 4, 2013 at 2:21
• @cardinal Yes, sorry, the unconditional means are constant with respect to $t$. For example, $X_t$ might be an AR(1) process with AR coefficient less than one in absolute value. I've re-worked the question to include some explicit assumptions on the population and sample moments of $X_t$ and $Y_t$. Let me know if anything is still unclear, or if you think the assumptions need to be stated more rigorously, and I will endeavour to do so. Cheers! Feb 4, 2013 at 3:13
• If $\mathbb{E} [Y_t | X_t] = X_t$, we have $Y_t = X_t + Z_t$, where $Z_t$ is a random variable with a density symmetric about zero. So, $\mathbb{Var}Y_t = \mathbb{Var} X_t + \mathbb{Var} Z_t + 2 \mathbb{Cov} (X_t, Z_t)$ $\implies \mathbb{E}Y_t^2 = \mathbb{E} X_t^2 + \mathbb{Var} Z_t + 2 \mathbb{Cov} (X_t, Z_t)$ Therefore, I don't know how one can obtain a reasonable estimate of $\mathbb{E} X_t^2$ without making some assumptions about $Z_t$. Do you know anything about the conditional distribution of $Y_t$ given $X_t$? Feb 5, 2013 at 16:44
• @Innuo Thanks for your interest. Preferably, I would not make any assumptions regarding $Z_t$ other than $\mathbb{E}[Z_t|X_t]=0$. However, if it is necessary to get some traction, I can assume $\text{cov}(X_t,Z_t)=0$ and could possibly get away with assuming $Z_t$ is Normal. Unfortunately, in my application, estimating $\mathbb{V} Z_t$ accurately is difficult. Feb 5, 2013 at 23:45

I'm not sure you can do much more with what you've got to work with.

I mean, you can lower bound $\gamma \geq \mu^2$, as

$$\mathbb{Var}(X_t) = \mathbb{E}(X_t^2)-\mathbb{E}(X_t)^2 = \gamma-\mu^2 \geq 0$$

then estimate $\mu$ from the arithmetic mean of your set of samples $S_T$ as usual.

If we have the situation where $Y_t$ is equal to $X_t$ plus some uncorrelated noise, then you can upper bound it too as

$$\mathbb{E}(Y_t^2) \geq \mathbb{E}(X_t^2).$$

To see why, look at Innuo's splendid comment on your original post, and note that the variance of $Z_t$ must be positive. If the covariance between it and $X_t$ is zero, then the inequality becomes pretty clear. If the correlation between $X_t$ and $Z_t$ is positive, then it still holds, albeit more weakly.

So we have $\mathbb{E}(Y_t^2) \geq \gamma \geq \mu^2$ for uncorrelated or positively correlated additive noise. I think that's all that can be said. If those bounds are sufficiently tight then maybe that will do? If not, then either $X_t$ or $Z_t$ has a big variance, and beyond taking more direct measurements somehow I'm not sure if you can tell which.

• Thanks for the response, I was worried that this might be the case. +1 for elucidating the bounds so clearly - if no one else has answered by the end of the bounty period, then I'll give the tick to this answer. Cheers. Feb 8, 2013 at 0:18