2
$\begingroup$

Given a data sample $\{x_i\}_1^n$, instead of hard omitting outliers by e.g. trimming, one can form a weighted average where we soft penalize observations out in the tails.

\begin{align} \mu = \frac{\sum_{i=1}^n w_ix_i}{\sum_{i=1}^n w_i} \end{align}

Now the question is the choice of $w_i$.

One method is the distance-weighted mean, where observations that have large distances to other observations get penalized more, according to \begin{align} w_i = \frac{1}{\sum_{k=1}^n |x_i-x_k|^p}, \end{align} where $p=1$ for euclidean distance, and $p=2$ for squared distance. Obviously, using $p=2$ penalizes non-central data points more.

Let us now "invent" the implicit distance-weighted mean, where observations that have a large distance to the to be estimated mean get penalized more, according to \begin{align} w_i = \frac{1}{|x_i-\mu|^p}. \end{align} Since we are seeking $\mu$, but as it also is included in the weights, we can find it by ways of optimization.

Question: Ignoring issues with the implicit method like convergence and numerical issues, what are some different properties of these two methods, and when do one prefer one over the other (especially for small sample size)? I have not found any info on the implicit method, and I wonder if it even makes sense.

Improve this question
$\endgroup$
5
  • 1
    $\begingroup$ Think about two data points and $p=2$, or any symmetric data set really. Intuitively (i.e., I may be wrong because I haven't spent much time thinking about it myself), wouldn't the algorithm stick at the initial $\bar{x}$? $\endgroup$
    – jbowman
    Commented Apr 25, 2022 at 15:03
  • 2
    $\begingroup$ IDW usually concerns spatial data $(x_i, z_i)$ and predicts $z$ at any location $x$ using the values of $z_i$ weighted by inverse powers of the distances $|x-x_i|.$ You ask about the prediction at the point $x=\bar x$ of the data $(x_i, x_i).$ Intuitively it will be extremely close to the mean itself. It's certainly not robust, since it depends on the non-robust mean. As such, your algorithm is nearly circular: you don't really accomplish anything apart from nudging the mean a tiny bit. When you iterate it, likely little will happen. What, then, is the point? What do you hope to achieve? $\endgroup$
    – whuber ♦
    Commented Apr 25, 2022 at 15:49
  • $\begingroup$ @whuber: for $p=1$, for odd number of data points, the solution is the median. For even number, take the observations {0, 1, 2, 10}, which has arithmetic mean 3.25 and median 1.5. Using artithmetic mean as initial guess, the solution is 1.94. That is, it is not near the mean? However, the solution isn't unique since it seems to depend on the initial guess. I will find out more about p=2. $\endgroup$
    – Pontus Hultkrantz
    Commented Apr 26, 2022 at 14:07
  • 1
    $\begingroup$ It sounds like you are not iterating, as your post seems to say, but optimizing. That's totally different--and has almost nothing to do with IDW. Please edit your post to clarify what you're trying to ask. $\endgroup$
    – whuber ♦
    Commented Apr 26, 2022 at 14:30
  • $\begingroup$ @whuber: correct, thanks. I will post an answer myself. The new method is totally useless, for any p. In the p=1 case, solution is any data point, and for even number of observations any point between the two middle data points is valid. Same applies to p=2 case, but it also has solutions that fall between data point. $\endgroup$
    – Pontus Hultkrantz
    Commented Apr 26, 2022 at 15:25

2 Answers 2

Reset to default
0
$\begingroup$

The so-called "implicit distance-weighted mean", not to be confused with IDW used for interpolation, is not a feasible way of estimating any form of robust mean. The reason is the following

  • It does not have a unique solution. In fact, any one of the original data points will always serve as a solution. This is due to the reciprocal of a zero distance causing the weight to explode. This is regardless of $p$,

In addition,

  • In the special case when $p=1$ and the number of data points $n$ is even, any point $\in\mathbb{R}$ in between the two data points neighbouring the median will also belong to the set of solutions.
  • When $p>1$, there will be one solution between each neigbouring data points, so an extra $n-1$ solutions, for a total of $2n-1$ solutions.
Improve this answer
$\endgroup$
0
$\begingroup$

Here's a robust weighted mean I came up with

$$w_i = \sum_j e^{-(x_i - x_j)^2}$$

Don't know if it's in the literature.

Probably not scale invariant, but it seems to solve my problem.

Improve this answer
$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.