0
$\begingroup$

I have data with a tail to the right (gamma/lognormal/weibull dist.) I use the package abodOutlier to which returns angle-based outlier factor for each observation. A small abof respect the others would indicate presence of an outlier. This is the distribution of the abof:

ABOF Histogram

Another plot of ABOF

The range of the abof is between 1.103859e-10 and 9.565497e-01 which is 0.0000000001103859 to 0.9565497... So my question is what interval should i consider as "outliers"? Is there anything standard for this method (say from 0 to 0.2 are outliers, or something along those lines) or should i just develop this interval specifically based on the company's interest and sensitivity?

$\endgroup$
  • 1
    $\begingroup$ What is an outlier is a matter of perspective. You should set it so that it answers or completes a specific task you are trying to achieve. Sometimes there are "standards" but they're not necessarily the best. $\endgroup$ – Roman Luštrik Dec 19 '16 at 9:49
  • $\begingroup$ @RomanLuštrik Well... The data consists of gross sales revenue and net sales revenue as 2 variables, for a specific country, channel and even type of customer. So really, the task is to identify which customers are potential outliers and investigate the processes in between GSR and NSR. This is given the fact that according to the company policy all customers in a channel are of the same type and receive the same conditions. So the GSR and NSR per customer in a given channel should be relatively the same. Finding points that don't fit the channel is my point. I hope this is understandable :-D $\endgroup$ – Emil Filipov Dec 19 '16 at 10:43
  • $\begingroup$ It depends on your data and your problem. However, it may help to graph the data with plot(sort(abof)) Does the data shift continuously from 0.95 down to 0 or are there gaps? This is a tool that will help you understand your data, but you must still do the understanding part. $\endgroup$ – G5W Dec 19 '16 at 11:03
  • $\begingroup$ @G5W Thank you for the tip about the plot. I have updated the question with the plot. You can say that there are more serious gaps in the abof starting at around 0.3. Other then that i think the gaps above 0.5 are not really significant as the factor is "high" i guess.. Mainly my concern is what is considered high and low regarding this factor. The data might be smoothly sloping down, but that doesn't mean that there are no outliers. After all abof is a coefficient of the relative position of a point to the other points (i think) the lower it is, the "further" away the point is from the pool? $\endgroup$ – Emil Filipov Dec 19 '16 at 11:38
0
$\begingroup$

@user7261442 I don't think that is the right interpretation. If a point is far away from others, you would expect a low abof, but not vice versa. Low abof means that the range of angles to other points is small. That can happen in other ways. Look at this.

Low ABOF

The points off to the right can hardly be called outliers, but will have low ABOF. By stretching this picture out, you can generate non-outliers with arbitrarily small ABOF.

I don't think that you can set a threshold for all data sets and say "Below this is an outlier". But when there are gaps in the ABOF curve, points on the low side of the gap are likely different from those above the gap.

| cite | improve this answer | |
$\endgroup$
  • $\begingroup$ You are right. There could be non-outliers with a small ABOF. Then again it depends on the definition of an outlier one is using. Again, thank you for the answer. $\endgroup$ – Emil Filipov Dec 19 '16 at 14:49
  • $\begingroup$ Regarding the gaps i will be sure to use that somehow and investigate the data above and below the gaps. $\endgroup$ – Emil Filipov Dec 19 '16 at 14:51

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.