0
$\begingroup$

Is it possible to exist outliers in binary columns? At first I thought it was impossible based on this Outlier detection for a univariate categorical variable? , assuming that binary columns also mean categorical data in this example.

But then, while working on a dataset on kaggle, I saw this notebook: https://www.kaggle.com/bariscal/stroke-entirely-ml-project-and-eda/comments

In the Anomaly Detection section, the author removed rows where IQR (InterQuartile Range) identified their respective columns as containing outliers, among those columns there were binary data (categorical), such as a column that said if a person was or wasn't married, and even the target variable (the column the author was trying to predict) that was also a binary column.

Can someone clarify this to me? Is it correct to remove outliers from a binary column like that?

Improve this question
$\endgroup$
11
  • 3
    $\begingroup$ I'm not going to read that notebook, but in an univariate binary situation the only way to see an observation as outlier is for it to belong to a category with very, very low probability. Even then it is strange because the fact that the category is there should normally mean that it's legitimate to observe something there. However, with more than one variable there can surely be outlying multivariate binary observations with very rare combinations of values that "lie out" according to a suitable distance measure. $\endgroup$
    – Christian Hennig
    Commented Mar 29, 2021 at 23:01
  • 1
    $\begingroup$ Ok, this is really silly territory. You mean delete everything outside the central 50%? For continuous data that would be stupid to put it mildly. For binary data what does it mean? Randomly picking 50% of the 0's and 1's and deleting them? Or deleting all 0's and 1's when the percentage is less than 25%? I can't see any way of describing this whole enterprise other than utter BS. $\endgroup$
    – BigBendRegion
    Commented Mar 30, 2021 at 11:06
  • 1
    $\begingroup$ The problem of course may be that the thread opener has misunderstood the notebook, or that the notebook is written in a misleading way... $\endgroup$
    – Christian Hennig
    Commented Mar 30, 2021 at 13:30
  • 1
    $\begingroup$ The question misrepresents the procedure: it uses the fences in a boxplot to detect outliers, not the IQR itself. This remains a counterproductive way to deal with binary data, though, because it will eradicate any values that have a frequency of less than 25%. Maybe the best solution is to ignore that notebook and move on. And yes, there are legitimate, useful concepts of "geometric" or "multivariate" binary outliers which stand out from most other data due to combinations of values. See stats.stackexchange.com/questions/108741, for instance. $\endgroup$
    – whuber ♦
    Commented Mar 30, 2021 at 16:08
  • 1
    $\begingroup$ @BigBendRegion I believe this is a platform build to help people better understand problems, just because a concept might not make sense to you doesn't mean is BS. By IQR I meant the procedure done in this article towardsdatascience.com/… feel free to check it if it's not stupid to you. $\endgroup$
    – Caldass_
    Commented Mar 31, 2021 at 0:17

0

Reset to default

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.