I am working with Classification Machine Learning problems and have come across a problem where I have 0 IQR for my data. No matter what technique I use, square root, cube root , log transform I still get the same IQR (obviously it is used to scale the data uniformly).

How do I get rid of outliers in my data. Below is the Value Counts for the data.

Amount  Number of People
360.0    843
180.0     66
480.0     23
300.0     20
240.0      8
84.0       7
120.0      4
36.0       3
60.0       3
12.0       2
350.0      1
6.0        1

In this data, Q1=Q3= 360 making IQR=0

  • 1
    $\begingroup$ No one to one transformation can separate equal values. If 843 values (86%) are together at 360, they will stay together at whatever you transform 360 to. $\endgroup$ – Glen_b -Reinstate Monica Dec 19 '19 at 3:22
  • 1
    $\begingroup$ If you want to base your definition of being an outlier on the IQR, such as median $\pm 1.5*IQR$, then you have your answer: everything but 360 is an outlier. $\endgroup$ – Dave Dec 19 '19 at 3:33
  • 1
    $\begingroup$ What are you doing with your data set once you examine it for outliers? $\endgroup$ – Dave Dec 19 '19 at 3:43
  • 1
    $\begingroup$ Without a specific model and purpose, what would make something an outlier, and why would you need to identify them? It sounds like you don't have a useful definition of "outlier" $\endgroup$ – Glen_b -Reinstate Monica Dec 19 '19 at 3:57
  • 1
    $\begingroup$ And why do you want to eliminate data points? “Outliers” can be quite telling. (Really, I call an outlier a mistake, like an intern entering $6.0\times 10^{9}m$ instead of $6.0\times 10^{-9}m$ when you know you’re working at a nanometer scale. I prefer to use “extreme value” to describe a real observation that is outside the norm.) $\endgroup$ – Dave Dec 19 '19 at 4:11

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Browse other questions tagged or ask your own question.