The observations in my dataset can be split in two classes. The observations in class 1 are for sure correctly labeled. The observations that has been designated to class 2 have a huge percentage of mislabeled data. I also know that there are as much mislabeled data in class 2 as there are observations in class 1. As an example I did a small simulation of a potential dataset, to simplify things I only simulated 1 variable and I just copied class 1 observations as the mislabeled class 2 observations. (Note that the true dataset is much larger and contains multiple variables but the problem remains the same and this simulation captures the essence.)
Class1 = rnorm(1000,0) Class2 = c(rnorm(1000,10),Class1) Class1 = data.frame(score = Class1,Class1 = TRUE) Class2 = data.frame(score = Class2,Class1 = FALSE) dat = rbind(Class1,Class2)
My goal is to decide which observations in class 2 are truly from class 2 and which should have been in class 2. Since I have multiple variables I decided to fit a logistic regression model on my data and use the probabilities as a metric to decide if the observations of class 2 are mislabeled or not.
fit = glm(Class1~.,data= dat, family = binomial) hist(fit$fitted.values,xlim = c(0,1),xlab = "P(Class 1)", main='Histogram of probabilities of being in Class 1' )
If you plot the probabilities in the histogram, you see a lot of observations around 0. These are probably the true class 2 observations. You can also see a lot of observations around .5. This makes sense. You know that there are as many observations in class 1 as there are mislabeled observations in class 2. So if you see an observation with a parameter value that comes from the 'class 1' distribution, you can have 50 percent probability that is has a class 1 label and 50 percent probability that is has an class 2 label (so mislabeled).
But I find the above distribution of probabilities shown in the histogram somehow peculiar. I would have expected that all values would be between 0 and 0.5. Instead I also see some values for example around .6. So if I see an observations with these variable values I'm 60 percent sure it belongs to class 1. How can this be? In my simulation example I just copied the same values for class 1 and the mislabeled class 2, so above observation should in theory not be possible.
How should I interpret this? Are these probabilities meaningful?
EDIT: In response to @whuber I also added the following plot of the data.
plot(dat); curve(1/(1 + exp(-coef(fit) %*% rbind(1, x))), add=TRUE)