I am trying to find the optimal estimator for the maximal expected $\Sigma X_i$ where $X_i$ is sampled from an unknown distribution which is chosen to be maximal. To clarify and simplify, there are two sets of data $Y_i$ and $Z_i$ which are drawn from similar but likely different pdf distributions $y$ and $z$. I would like an estimator on $Z_i$ and $Y_i$ which would predict which pdf would produce a larger sum over a future sampling, ie $\Sigma X_i$.
The number of elements in each set $N_y$ and $N_z$ is chosen by our implementation so we have to normalize the number of elements out. Or put another, way we are more interested in the mean $\mu = \Sigma X_i / N_x$ because our future sampling has a fixed $N_x$ independent of the choice of $y$ or $z$. This leads to the standard method of making a choice based on each scenario's mean, $\mu$, and error on mean $\sigma_\mu$. What I am interested in is if the mean is really the best estimator to choose the scenario which leads to the highest expected $\Sigma X_i$. In a simple scenario I am sure the answer is yes but there are some complications.
There is no good model for the underlying distributions which produces $Y_i$ or $Z_i$ but a histogram of either decreases exponentially. Hence the occurrence of a few high large value outliers which make things complicated. The main reason I think the mean may not be the best is because of its sensitivity to these outliers. The solution of removing outliers and doing a bootstrapping method is discussed here: Bootstrapping - do I need to remove outliers first? From this I think it is clear that the removal of outliers is ill advised. Additionally, the number of elements, $N_y$ and $N_z$, is in the millions so the CLT asserts we do not need to bootstrap at all to obtain $\sigma_\mu$.
Another complication is that the distribution is quantize/discrete in two ways. To illustrate this lets take the example of optimizing for future revenue in a split test. There are two scenarios that lead to two sets $Z_i$ and $Y_i$ representing the amount spent for each customer. To illustrate that the underlying pdfs are not continuous one must first off note that the smallest unit of money is a penny so the range of each pdf is the integers. Secondly and more importantly the items to buy do not span the space. Let say for example there is one item that costs \$20 but no other items in the price range [\$15,\$25]. Since there are a lot of single purchase customers there will be a peak at \$20.
The simplest choice of an alternative estimator would be the median, $\tilde{x}$ since it is less sensitive to outliers and insensitive to reparameterizations. However, since greater than 95% of customers spend \$0 the median will always be \$0. Also, I am not sure the median is a good estimator of the expected $\Sigma X_i$ in general.
So the question is, is there a better estimator than the mean of $Y_i$ and $Z_i$ for maximum expected $\Sigma X_i$? The word "better" in the context refers to the sense of gambling where the best bet is the best.
