3
$\begingroup$

I have a dataset with $m$ observations and $p$ categorical variables (nominal), each variable $X_1,X_2...X_p$ has several different classes (possible values).

Ultimately I am looking for a way to find anomalies i.e. to identify rows for which the combination of values seems incorrect with respect to the data I saw so far.

So far I was thinking about building a model to predict the value for each column and then build some metric to evaluate how different the actual row is from the predicted row. I would greatly appreciate any help!

Improve this question
$\endgroup$
2
  • 4
    $\begingroup$ Can you please share an example of your data and one row you think should be labeled as an outlier? $\endgroup$
    – yoav_aaa
    Commented Mar 18, 2019 at 12:50
  • $\begingroup$ How big is P? And how many observations have you got? $\endgroup$
    – Peter Flom
    Commented Jul 14, 2023 at 23:52

1 Answer 1

Reset to default
1
$\begingroup$

You could make one-hot-encodings of your categories, and then run an auto-encoder, and finally look at reconstruction errors. Those samples with the highest reconstruction errors are the most likely to be "abnormal". For an example of a workflow using H2O you can look at the following blog post.

Improve this answer
$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.