This approach has been proven to work in many realistic settings with substantive theory for models with error in every predicted probability output for every example.

[![enter image description here][1]][1]

The approach counts up an unnormalized estimate of the joint distribution of true labels and noisy/given labels. Using that estimate, it finds the label errors in the dataset so you can train on clean data. It has been shown to compare favorably to most methods and  works for any dataset you can train a classifier on and for most data formats, ML and deep learning frameworks, and data modalities, e.g. image, text, tabular, and audio data. I am an author on this package.

 - link to python package: https://github.com/cleanlab/cleanlab

## Find label issues in 1 line of code

```python
from cleanlab.classification import CleanLearning
from cleanlab.filter import find_label_issues

# Option 1 - works with sklearn-compatible models - just input the data and labels ツ
label_issues_info = CleanLearning(clf=sklearn_compatible_model).find_label_issues(data, labels)

# Option 2 - works with ANY ML model - just input the model's predicted probabilities
ordered_label_issues = find_label_issues(
    labels=labels,
    pred_probs=pred_probs,  # out-of-sample predicted probabilities from any model
    return_indices_ranked_by='self_confidence',
)
```

## Train a model as if the dataset did not have errors -- 3 lines of code

```python
from sklearn.linear_model import LogisticRegression
from cleanlab.classification import CleanLearning

cl = CleanLearning(clf=LogisticRegression())  # any sklearn-compatible classifier
cl.fit(train_data, labels)

# Estimate the predictions you would have gotten if you trained without mislabeled data.
predictions = cl.predict(test_data)
```

 - Journal of AI Research (with theory to prove it works): https://arxiv.org/abs/1911.00068publication
 - errors found using cleanlab: https://labelerrors.com/
 - Documentation and runnable tutorials for cleanlab: https://docs.cleanlab.ai/


  [1]: https://i.sstatic.net/oFox3.png