This seems like a a case of PU learning — learning from positive-only and unlabeled examples. The original paper on this is here. One of the main results is the following:
Suppose that $y$ is the true class label (0 or 1), $s$ is the observed class label (0 if negative or unlabeled, 1 if positive AND labeled), and $x$ is our data. Then $p(y = 1 | x) = p(s = 1 | x) * c$, where $c$ is a constant. In fact, you can show that $c = p(s=1 | y=1)$.
So, how do we apply this in the real world? Well, based on our dataset, we can make the following statements for all $x$:
- $p(s = 1 | y = 1; x) = \alpha$ (assume this is a constant)
- $p(s = 0 | y = 1; x) = 1 - \alpha$
- $p(s = 1 | y = 0; x) = 0$ (all labeled examples must be positive)
- $p(s = 0 | y = 0; x) = 1$ (you don't ever label a negative example)
Through some probabilistic manipulation, we can show that $\alpha = \frac{1}{c}$. So our problem is really about estimating $\alpha$. Suppose you had a magic function $h(x) = p(y=1 | x)$; i.e. you have the classifier that you want to learn. If you were to prove this in real life, one way to do this is to show that $\mathbb{E}[h(x) | y=1] = \alpha$. From this, we have the following algorithm:
$\alpha \sim= \frac{1}{|V^+|} \sum_{x \in |V^+|} h(x)$.
This means that you can naively train a model to discriminate between labeled positive examples and everything else, then rescale via $\alpha$ to obtain a decent estimator. That is; you don't have magical function $h(x)$, but the naive model behaves the same way on positive labeled examples as the ideal magical model.
Concretely, in a logistic regression toy example with $x \in \mathbb{R}^2$, if you draw the decision boundaries obtained from a naive vs. rescaled classifier, you will notice that this rescaling "shifts" the decision boundary linearly.
