Skip to main content
added 10 characters in body
Source Link
Tim
  • 141.2k
  • 26
  • 270
  • 512

Though it's a late reply, I'd like to point of implicit assumptions by previous answers that likely don't hold.

  1. for one-class classification, we don't know the real ratio of positive and negative data. So we cannot any development set has similar distribution to the real data.
  2. A standard setting for one-class classification is we have positive and unlabelledunlabeled dataset. We can't assume we have the label for "negative" data even in the development set. Also, we can't assume all the unlabelled data are "negative".

An alternative evaluation is proposed in the following paper (section 4):

Lee, Wee Sun, and Bing Liu. "Learning with positive and unlabelled examples using weighted logistic regression." ICML. Vol. 3. 2003.

They uses

r^2/Pr[Y=1]

$$ \frac{r^2}{\Pr(Y=1)} $$

P.S.: Prof. Lee, Prof Liu and Dr. Cheng are the people that coined one-class classification. We can take their evaluation as somewhat "official".

Though it's a late reply, I'd like to point of implicit assumptions by previous answers that likely don't hold.

  1. for one-class classification, we don't know the real ratio of positive and negative data. So we cannot any development set has similar distribution to the real data.
  2. A standard setting for one-class classification is we have positive and unlabelled dataset. We can't assume we have the label for "negative" data even in the development set. Also, we can't assume all the unlabelled data are "negative".

An alternative evaluation is proposed in the following paper (section 4):

Lee, Wee Sun, and Bing Liu. "Learning with positive and unlabelled examples using weighted logistic regression." ICML. Vol. 3. 2003.

They uses

r^2/Pr[Y=1]

P.S.: Prof. Lee, Prof Liu and Dr. Cheng are the people that coined one-class classification. We can take their evaluation as somewhat "official".

Though it's a late reply, I'd like to point of implicit assumptions by previous answers that likely don't hold.

  1. for one-class classification, we don't know the real ratio of positive and negative data. So we cannot any development set has similar distribution to the real data.
  2. A standard setting for one-class classification is we have positive and unlabeled dataset. We can't assume we have the label for "negative" data even in the development set. Also, we can't assume all the unlabelled data are "negative".

An alternative evaluation is proposed in the following paper (section 4):

Lee, Wee Sun, and Bing Liu. "Learning with positive and unlabelled examples using weighted logistic regression." ICML. Vol. 3. 2003.

They uses

$$ \frac{r^2}{\Pr(Y=1)} $$

P.S.: Prof. Lee, Prof Liu and Dr. Cheng are the people that coined one-class classification. We can take their evaluation as somewhat "official".

Source Link

Though it's a late reply, I'd like to point of implicit assumptions by previous answers that likely don't hold.

  1. for one-class classification, we don't know the real ratio of positive and negative data. So we cannot any development set has similar distribution to the real data.
  2. A standard setting for one-class classification is we have positive and unlabelled dataset. We can't assume we have the label for "negative" data even in the development set. Also, we can't assume all the unlabelled data are "negative".

An alternative evaluation is proposed in the following paper (section 4):

Lee, Wee Sun, and Bing Liu. "Learning with positive and unlabelled examples using weighted logistic regression." ICML. Vol. 3. 2003.

They uses

r^2/Pr[Y=1]

P.S.: Prof. Lee, Prof Liu and Dr. Cheng are the people that coined one-class classification. We can take their evaluation as somewhat "official".