Supervised learning with uncertain data? Is there an existing methodology for applying a supervised learning model to a uncertain dataset? For example, say we have a dataset with classes A and B:
+----------+----------+-------+-----------+
| FeatureA | FeatureB | Label | Certainty |
+----------+----------+-------+-----------+
|        2 |        3 | A     | 50%       |
|        3 |        1 | B     | 80%       |
|        1 |        1 | A     | 100%      |
+----------+----------+-------+-----------+

How could we train a machine learning model on this? Thanks.
 A: Instead of having labels A or B, you could replace them with continuous values of the certainty -- for example, $1$ corresponds to something you're sure is $A$, $0$ corresponds to something you're sure is $B$ and $0.6$ corresponds to something you're 40% sure is $A$. Then, have a model that instead of predicting class $A$ or $B$ outputs a score between $0$ and $1$ based on how much you think its one or the other (and threshold this score based on if its > or < 1/2). This turns your classification problem into a regression problem (which you threshold to get back to a classifier).
For example, you could fit a linear model to $\log \frac{p(A|x)}{p(B|x)} = \log \frac{p(A|x)}{1-P(A|x)} $ as $ \beta_0 + \beta_1^T x $ (where $p(A|x)$ is the certainty above). Then, when you want to test some data, plug it into the model, and output label $A$ if $ \beta_0 + \beta_1^T x >0 $ and $B$ otherwise. 
A: As a numerical quality you ascribe to your data, I think this "certainty" could surely be used as a weight. Higher "certainty" scores increase the weight a datum has on the decision function, which makes sense.
Many supervised learning algorithms support weights, so you just have to find a weighted version of the one you intend to use.
