I am working on a project where the training labels are given to me as a probability value in the range [0,1]. My first approach was to fit a simple linear ridge regression to predict the probability. This isn't ideal as:
1) Predictions from this model end up with values outside the range of 0 and 1 2) I don't think linear regression works that well if you want to make it predict something in a fixed interval.
I try the logit transformation, but since I have probabilities that are exactly 0 and 1, I perturb them slightly so I get 0.00001 and .999999 instead to avoid +/- infinity. I train my model on the transformed labels, and then make a prediction on the test set and undo the transformation with the inverse of the logit function (logistic function). Frustratingly though, this gives me even worse results than the naive linear regression!
Any suggestions on other transformations I can try or what I am doing wrong?