0
$\begingroup$

I was searching outlier removal in R and I saw some comments related to almost never you should remove outlier from dataset. I wonder when we should remove outlier? I have a dataset consisting outliers because of over pricing of sellers in house prices. I think they are making my data noisy.Should I remove them with R programming? I know that if I have large enough dataset maybe they become less important right? but now I have a small dataset with categorical and numeric values. And which methods in R I can use? Thank you

$\endgroup$
7
  • 1
    $\begingroup$ I don't think that this is the right forum to answer this question. If and how to remove outliers is a rather complex topic that is discussed in books on data mining. As far as I know the decision is often subjective and it usually depends one the problem that is considered. In my opinion there is no cookie-cutter solution and it is generally not a matter of programming. $\endgroup$ – RHertel Jul 31 '15 at 10:25
  • $\begingroup$ Yes some people say that outliers shouldn't be removed from dataset some of them we should it is subjective. However, I saw these discussions at stackoverflow and I want to ask here because of that. @RHertel $\endgroup$ – tyer Jul 31 '15 at 10:40
  • 1
    $\begingroup$ Since you have "outliers because of over pricing of sellers in house prices", I would suggest that you should use a model that can represent over-pricing. $\endgroup$ – Roland Jul 31 '15 at 11:28
  • $\begingroup$ I didn't understand exactly Can you give example? Do you mean I should use robust regression? @Roland $\endgroup$ – tyer Jul 31 '15 at 11:40
  • 1
    $\begingroup$ Robust regression might be an option, but I meant that you could use a model that explicitly models over-pricing. Sorry, I don't have time to create an example. I would consider that your job anyway. $\endgroup$ – Roland Jul 31 '15 at 11:46