I have a dataset with $m$ observations and $p$ categorical variables (nominal), each variable $X_1,X_2...X_p$ has several different classes (possible values).

Ultimately I am looking for a way to find anomalies i.e. to identify rows for which the combination of values seems incorrect with respect to the data I saw so far.

So far I was thinking about building a model to predict the value for each column and then build some metric to evaluate how different the actual row is from the predicted row. I would greatly appreciate any help!

  • 2
    $\begingroup$ Can you please share an example of your data and one row you think should be labeled as an outlier? $\endgroup$ – yoav_aaa Mar 18 '19 at 12:50

You could make one-hot-encodings of your categories, and then run an auto-encoder, and finally look at reconstruction errors. Those samples with the highest reconstruction errors are the most likely to be "abnormal". For an example of a workflow using H2O you can look at the following blog post.


Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.