This question is a bit generic.

Let's suppose we have some quantities measured $X_i$ and we want to estimate some quantity, for example $\mu$, the mean of the distribution.

For every quantity we also have a way to estimate the probability of $X_i$ being an outlier, $p_i$ dependent on an other parallel measurement not involving the quantity if interest, in this case the mean of the distribution (e.g. measurements involving higher moments...). Is there a standard way to incorporate this information in a rigorous way in the analysis?

For example if we want to estimate the mean of the distribution one could try something like: $\sum_i p_i X_i$ (and normalize). Is there a standard, Bayesian maybe, way to make such an estimate rigorous? Do you have references of examples where in a particular setting such an analysis is performed? At the end such things are done every time someone identifies and removes by hand outliers.

  • $\begingroup$ It is difficult to see how you could make any progress knowing only the "probability of being an outlier." To begin with, what would the definition of "outlier" be? Second, in order to use this probability at all, you would need to know the distribution of non-outliers. $\endgroup$ – whuber Mar 28 '19 at 18:15
  • $\begingroup$ Thx for the answer. $p_i$ could be identified for example from a second independent statistics and identifying, from the separate measurement, the probability of being an outlier according to its following a common distribution or not. The problem is that this second statistics can be not connected to the first quantity we want to estimate and in such a setting I wonder how this information can be introduced in the analysis and if there are standard techniques. $\endgroup$ – Thomas Mar 28 '19 at 18:29
  • $\begingroup$ Maybe some resampling technique could do the job? $\endgroup$ – Thomas Mar 28 '19 at 18:44
  • $\begingroup$ It isn't actionable information. Take an extreme example: I have one value, equal to $0.$ You tell me it has some probability of being an "outlier." Let's suppose that probability is $1.$ As far as you have indicated, all I can infer is that the value isn't $0$--but you have provided no information about what else it possibly could be. $\endgroup$ – whuber Mar 28 '19 at 18:53
  • $\begingroup$ Perhaps you should try some robust (regression) method less sensitive to the presence of outliers. $\endgroup$ – dnqxt Mar 28 '19 at 19:17

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Browse other questions tagged or ask your own question.