# Determine significance of an observed value

I have an ECDF of values that do not follow a particular distribution (thought they are slightly normal, they are not). And I wish to determine if a new observed value is significant or an outlier or not. How can I do this?

For example, I have the following distribution:

Value         % of Observations
...
-4                   3%
-3                   3.5%
-2                   4%
-1                   6%
0                   12%
1                    5%
2                    5%
3                    4%
4                    1%
...



With different distributions you set particular cutoffs or thresholds to signify an outlier, e.g. 3$$\sigma$$ for a normal distribution, but that doesn't help you classify ordinary values like a $$2$$ in the above case. Only 5% of observed values are $$2$$ but it's still quite common relative to the rest.

Is there some way to quantify the "outlierness" of a value? For example, if the value $$10$$ was observed I could say it is greater than $$99\%$$ of values possibly making it an outlier. However this won't work for non-outlier values, e.g. the value $$0$$ is greater than $$\approx50\%$$ of all observations but this doesn't tell me that $$0$$ is the most common value.

Note: I am not interested in fitting a particular distribution or anything. I just have a large dataset for which an ECDF can be evaluated and I want to know if a new observed value fits into this typically or is an extreme value.

• Your 'data' are impossible to interpret without knowing proportions of values outside $(-4,4)$ and without any idea of sample size. // Many distributions typically show 'outliers' (according to various criteria). Some characteristically show a large number of 'outliers' in any sample of moderate size. An observation should be deleted from a sample only if there is external evidence that it is impossible or an obvious error. (Examples: Man 5 meters tall, negative concentration, documented as data entry error or due to failure of measurement device.) Oct 5 '19 at 18:58
• This is a familiar question: search our site for "prediction interval."
– whuber
Oct 5 '19 at 19:52
• @BruceET I am not looking to delete observations and I have thousands of observations making up the above distribution. I just want to get some idea of how rare/extreme a new observation is relative to the thousands others I've seen already.
– guy
Oct 5 '19 at 22:44
• @whuber can you give a bit more info. Prediction intervals are more used in regression as far as I know and for prediction purposes. I am not trying to predict anything. I am just trying to compare how unlike or like a new observation is to a distribution that I already have.
– guy
Oct 5 '19 at 23:38
• A little like that. That's about the best you can do unless you have a more detailed model of how the values might be changing over time, in which case the concept remains the same but you would construct the prediction interval using a regression or time series procedure or whatever is appropriate for the model.
– whuber
Oct 6 '19 at 17:12