# Unbiased estimator of $X^k$ given independent unbiased estimators of $X$

Suppose we are interested estimating $$X^k$$, and we have access to independent unbiased estimators $$Y_i$$ for $$i = \{1,2,\dots, k\}$$, i.e., $$\mathbb{E}[Y_i] = X$$. A straightforward way to construct an unbiased estimator of $$X^k$$ is to define $$Y^{(k)} := \prod_{i=1}^kY_i$$, then $$\mathbb{E}[Y^{(k)}]=X^k$$ by independence.

What are other methods for estimating $$X^k$$ using only $$c < k$$ number of estimators? Is it even possible in this general setup? Is there a name to this type of problem?

• I'm reasonably confident you can't do this with $c<k$, but I haven't yet been able to find a proof. It's pretty clear for $k=2$: you can't get an unbiased estimate of a variance from one observation. Commented Dec 24, 2020 at 5:08
• @ThomasLumley: Unless I am misunderstanding the question, I don't think $k$ is the number of observations, so it seems possible in theory to do this (assuming sufficient data).
– Ben
Commented Dec 24, 2020 at 7:21
• @Thomas On the contrary, in realistic settings you can obtain an unbiased estimate of the variance from a single observation. For example, a single observation of a Poisson$(\lambda)$ variable estimates the variance $\lambda$ and is unbiased.
– whuber
Commented Dec 24, 2020 at 14:37
• Concerning the question: what exactly do you mean by "access to"? For instance, if that means you can compute $Y_i$ for any subset of your data, then if you have at least $k$ observations you can apply $Y_1$ to any $k$-subset of them and compute the product, etc.
– whuber
Commented Dec 24, 2020 at 14:41
• @whuber. Yes and if you know the variance is 7 you can get an unbiased estimator with zero observations. It wasn't clear to me that the original post was interested in making those sorts of assumptions; maybe I misread it. Commented Dec 24, 2020 at 23:01

The obvious analogy here is to use independent estimators $$Y_1,...,Y_c$$ with expectations:

$$\mathbb{E}(Y_i) = X^{p_i} \quad \quad \quad \sum p_i = k.$$

In theory this is possible so long as you have a method to construct the required estimators for this problem. The independence requirement will generally make it impractical, since in most cases it will require you to partition your dataset to form the estimators seperately. Consequently, while it is possible to get an unbiased estimator by this method, it will tend to be a poor estimator (with high variance), owing to the fact that it uses a partition of the data where only a small amount of the data is used for each part of the estimator.

An example using IID data with zero mean: Suppose you have IID data $$X_1,...,X_n$$ from some distribution with zero mean and finite variance $$\sigma^2 < \infty$$, and you want to estimate $$\mathbb{E}(X^k)$$. In this case the sample mean and sample variance give unbiased estimators for the first and second raw moments:

$$\mathbb{E}(\bar{X}) = \mathbb{E}(X) = 0 \quad \quad \quad \mathbb{E}(S^2) = \mathbb{E}(X^2) = \sigma^2.$$

Now, suppose we partition our data into $$c < k$$ parts where we have at least two data points in each part (so that we can form the sample variance). Denote these partition parts by $$\boldsymbol{X}_{(1)},...,\boldsymbol{X}_{(c)}$$ and denote statistics using these samples with the corresponding subscripts. Now, choose some values $$p_1,...,p_c \in \{ 1,2 \}$$ with $$p_1 + \cdots + p_c = k$$ and form the estimator:

$$\text{Est} \equiv \prod_{i:p_i = 1} \bar{X}_{(i)} \times \prod_{i:p_i = 2} S_{(i)}^2 .$$

The partition of the data means that the parts are independent, so we have:

\begin{align} \mathbb{E}(\text{Est}) &= \prod_{i:p_i = 1} \mathbb{E}(\bar{X}_{(i)}) \times \prod_{i:p_i = 2} \mathbb{E}(S_{(i)}^2) \\[6pt] &= \prod_{i:p_i = 1} \mathbb{E}(X) \times \prod_{i:p_i = 2} \mathbb{E}(X^2) \\[6pt] &= \prod_{i:p_i = 1} \mathbb{E}(X^{p_i}) \times \prod_{i:p_i = 2} \mathbb{E}(X^{p_i}) \\[6pt] &= \prod_{i} \mathbb{E}(X^{p_i}) \\[6pt] &= \mathbb{E}(X^{\sum p_i}) \\[12pt] &= \mathbb{E}(X^k). \\[12pt] \end{align}

Note that although this is an unbiased estimator, it will be a terrible estimator when the number of partition pieces is large.

• Hi Ben, thank you for this answer! Btw, I updated the question for clarity, and revamped the notation. Naively, I want to try a bootstrap method that leverages the full (in your example) data set instead of using partitions. Would that still yield an unbiased estimator and reduce variance?
– fool
Commented Dec 24, 2020 at 20:58
• Well, it is unclear to me how you will preserve independence of the estimators if they share data points. Do you have a proposal for how to do this?
– Ben
Commented Dec 24, 2020 at 21:48
To illustrate the point that the answer depends on the underlying statistical model: If $$Y\sim\mathcal E(1/X)$$, an exponential variable, then $$\mathbb E[Y^k]= X^k\Gamma(k+1)$$ meaning that $$Y^k/\Gamma(k+1)=Y^k/k!$$ is an unbiased estimator of $$X^k$$, based on a single observation. This extends to Gamma variables, obviously.