# Estimate the second moment of a latent variable using a conditionally unbiased proxy

The Setup: Let $X_t$ denote an unobservable stochastic sequence where the first two unconditional moments are finite constants; ie $\mathbb{E} X_t = \mu < \infty$ and $\mathbb{E} X_t^2 = \gamma < \infty$. Assume that $X_t$ obeys some minimum set of regularity conditions such that a weak law of large numbers applies to its first two sample moments; ie conditions that limit the dependence in the sequence and bound the higher order moments.

Let $Y_t$ denote a conditionally unbiased proxy for $X_t$. That is, $\mathbb{E} [Y_t | X_t] = X_t$, and hence $\mathbb{E} Y_t = \mathbb{E} X_t = \mu$. Assume that $Y_t$ also obeys a minimum set of regularity conditions such that a weak law of large numbers applies to its first two sample moments.

Let $S_T = T^{-1} \sum_{t=1}^T Y_t$. Given the assumptions on $X_t$ and $Y_t$, it follows that $\mathbb{E} S_T = \mu$ and $S_T \overset{\mathbb{P}}{\rightarrow} \mu$, hence $S_T$ is a consistent, unbiased estimator for $\mu$

My Question: Given the setup, how can I estimate $\mathbb{E} X_t^2 = \gamma$?

Things I Know: From Slutsky's theorem, $(S_T)^2 \overset{\mathbb{P}}{\rightarrow} (\mathbb{E} X_t)^2 \neq \mathbb{E} X_t^2 = \gamma$, via Jensen's inequality, so in general $(S_T)^2$ will be a poor estimator for $\gamma$. Also, $T^{-1} \sum_{t=1}^T Y_t^2$ may not be a good estimator since the assumption set does not guarantee that $\mathbb{E} X_t^2 = \mathbb{E} Y_t^2$.

Final Note: I would be happy with any answer that improves upon the properties of $(S_T)^2$. So an approximation type estimator obtained via some sort of series expansion would be great. That's what I've mainly been looking into thus far, but with little luck. Even suggestions regarding expansions I might look into would be most welcome.

Thanks in advance to all responders.

• Hi, Colin. This question looks interesting, but I have several questions about it. Perhaps they are just notational issues or unstated assumptions, but much of the math in the question doesn't make complete sense. Are the means $\mu_t = \mathbb E X_t$ constant? (Writing something like $S_T \stackrel{p}{\to} \mathbb E X_t$ doesn't make sense, otherwise.) What other assumptions do you make, but, perhaps, haven't stated here? The more structure you can give to the problem, the better the potential answer. – cardinal Feb 4 '13 at 2:21
• @cardinal Yes, sorry, the unconditional means are constant with respect to $t$. For example, $X_t$ might be an AR(1) process with AR coefficient less than one in absolute value. I've re-worked the question to include some explicit assumptions on the population and sample moments of $X_t$ and $Y_t$. Let me know if anything is still unclear, or if you think the assumptions need to be stated more rigorously, and I will endeavour to do so. Cheers! – Colin T Bowers Feb 4 '13 at 3:13
• If $\mathbb{E} [Y_t | X_t] = X_t$, we have $Y_t = X_t + Z_t$, where $Z_t$ is a random variable with a density symmetric about zero. So, $\mathbb{Var}Y_t = \mathbb{Var} X_t + \mathbb{Var} Z_t + 2 \mathbb{Cov} (X_t, Z_t)$ $\implies \mathbb{E}Y_t^2 = \mathbb{E} X_t^2 + \mathbb{Var} Z_t + 2 \mathbb{Cov} (X_t, Z_t)$ Therefore, I don't know how one can obtain a reasonable estimate of $\mathbb{E} X_t^2$ without making some assumptions about $Z_t$. Do you know anything about the conditional distribution of $Y_t$ given $X_t$? – Innuo Feb 5 '13 at 16:44
• @Innuo Thanks for your interest. Preferably, I would not make any assumptions regarding $Z_t$ other than $\mathbb{E}[Z_t|X_t]=0$. However, if it is necessary to get some traction, I can assume $\text{cov}(X_t,Z_t)=0$ and could possibly get away with assuming $Z_t$ is Normal. Unfortunately, in my application, estimating $\mathbb{V} Z_t$ accurately is difficult. – Colin T Bowers Feb 5 '13 at 23:45

I'm not sure you can do much more with what you've got to work with.

I mean, you can lower bound $\gamma \geq \mu^2$, as

$$\mathbb{Var}(X_t) = \mathbb{E}(X_t^2)-\mathbb{E}(X_t)^2 = \gamma-\mu^2 \geq 0$$

then estimate $\mu$ from the arithmetic mean of your set of samples $S_T$ as usual.

If we have the situation where $Y_t$ is equal to $X_t$ plus some uncorrelated noise, then you can upper bound it too as

$$\mathbb{E}(Y_t^2) \geq \mathbb{E}(X_t^2).$$

To see why, look at Innuo's splendid comment on your original post, and note that the variance of $Z_t$ must be positive. If the covariance between it and $X_t$ is zero, then the inequality becomes pretty clear. If the correlation between $X_t$ and $Z_t$ is positive, then it still holds, albeit more weakly.

So we have $\mathbb{E}(Y_t^2) \geq \gamma \geq \mu^2$ for uncorrelated or positively correlated additive noise. I think that's all that can be said. If those bounds are sufficiently tight then maybe that will do? If not, then either $X_t$ or $Z_t$ has a big variance, and beyond taking more direct measurements somehow I'm not sure if you can tell which.

• Thanks for the response, I was worried that this might be the case. +1 for elucidating the bounds so clearly - if no one else has answered by the end of the bounty period, then I'll give the tick to this answer. Cheers. – Colin T Bowers Feb 8 '13 at 0:18