This approach has been proven to work in many realistic settings with substantive theory for models with error in every predicted probability output for every example. [![enter image description here][1]][1] The approach counts up an unnormalized estimate of the joint distribution of true labels and noisy/given labels. Using that estimate, it finds the label errors in the dataset so you can train on clean data. It has been shown to compare favorably to most methods and works for any dataset you can train a classifier on and for most data formats, ML and deep learning frameworks, and data modalities, e.g. image, text, tabular, and audio data. I am an author on this package. - link to python package: https://github.com/cleanlab/cleanlab ## Find label issues in 1 line of code ```python from cleanlab.classification import CleanLearning from cleanlab.filter import find_label_issues # Option 1 - works with sklearn-compatible models - just input the data and labels ツ label_issues_info = CleanLearning(clf=sklearn_compatible_model).find_label_issues(data, labels) # Option 2 - works with ANY ML model - just input the model's predicted probabilities ordered_label_issues = find_label_issues( labels=labels, pred_probs=pred_probs, # out-of-sample predicted probabilities from any model return_indices_ranked_by='self_confidence', ) ``` ## Train a model as if the dataset did not have errors -- 3 lines of code ```python from sklearn.linear_model import LogisticRegression from cleanlab.classification import CleanLearning cl = CleanLearning(clf=LogisticRegression()) # any sklearn-compatible classifier cl.fit(train_data, labels) # Estimate the predictions you would have gotten if you trained without mislabeled data. predictions = cl.predict(test_data) ``` - Journal of AI Research (with theory to prove it works): https://arxiv.org/abs/1911.00068publication - errors found using cleanlab: https://labelerrors.com/ - Documentation and runnable tutorials for cleanlab: https://docs.cleanlab.ai/ [1]: https://i.sstatic.net/oFox3.png