I want to clean my data but one variable has more than 86 percent zero and consequently iqr=0. When I want to clean outliers, that feature is eliminated because all the non-zeros were defined as outliers or extremes, and the value of that feature is equal to 0 for all of the records.
I think that is not reasonable, but how do I manage this situation?
I'll start with referring to your comment:
i am in preprocessing phase . and outlier should be removed in this phase
It is not true that during data preprocessing you need to remove the outliers. By any means, this is not a default approach. Yes, you should remove invalid data, but by invalid we mean the data that is corrupted, or incorrect (e.g. human age of 1000 years old). You do not remove the rare samples.
The problem described in your question is a great example of why we don't do this. In your case, removing the samples that occur in the data less frequently would mean that you dropped all the non-zero entries, leaving you with a column that is constant, hence useless, so in fact you would need to remove the whole column from your data. What a waste!
Moreover, the infrequent samples may be very valuable in terms of the unique information they bring to the data. Say that you are studying successes and failures of companies across the time. If you removed the companies that lasted the shortest amount of time, and the ones that lasted the longest among of time, you are losing a lot of valuable information. If you did a medical study and dropped the subjects that had some rare symptoms, it might be the case that you removed from your data the samples that could shed most light on the studied phenomenon.
I highly recommend that you check also other questions tagged as outliers to learn more. One of the things that you would learn from those threads is that even if you have good reason to remove the outliers, using some simple criteria like points outside of few standard deviations from the mean, or outside IQR range, in many cases are pretty bad ideas.
