Using this example on the scikit-learn site, I am attempting to do some anomaly detection using LOF. What I end up with is this:

clf = LocalOutlierFactor(n_neighbors=20, contamination='auto')
y_pred = clf.fit_predict(df)    

data_scores = clf.negative_outlier_factor_

My dataframe contains 47 columns. My immediate issue is that I'm not sure how to use data_scores to detect anomalies. (The example on the site above uses an 2-column dataframe, I believe, so it's a bit more straightforward.) For example, if I manually multiply a value in my data by 10, I can see a corresponding increase in the data_scores maximum absolute value. But what does that get me? It might help me to identify which row of the data contains an anomaly, but can it help me identify which field in that row? I'm probably missing something big-picture here, so feel free to point me in the right direction.

I'm hoping for a little guidance on topic(s) I might look into. The LOF examples online all seem to stop with the code I included above, assuming the user will know what to do with the numpy array the negative_outlier_factor_ method returns.


migrated from stackoverflow.com Aug 21 at 9:13

This question came from our site for professional and enthusiast programmers.

  • $\begingroup$ Consider the 2D example: even in that case, how would you visually identify which variable (that is, coordinate) is "anomalous"? In some cases you might be able to argue "only by changing $x$ could one move this point into a cluster," and thereby implicate the $x$ coordinate; but in other cases you could move the point into a cluster by changing either $x$ or $y$ and in some cases you would have to change both coordinates. Why, then, would it be meaningful to suppose there is any answer (in general) to the question "which field in that row?" $\endgroup$ – whuber Aug 26 at 21:31

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Browse other questions tagged or ask your own question.