I am working on a project where the training labels are given to me as a probability value in the range [0,1]. My first approach was to fit a simple linear ridge regression to predict the probability. This isn't ideal as:

1) Predictions from this model end up with values outside the range of 0 and 1 2) I don't think linear regression works that well if you want to make it predict something in a fixed interval.

I try the logit transformation, but since I have probabilities that are exactly 0 and 1, I perturb them slightly so I get 0.00001 and .999999 instead to avoid +/- infinity. I train my model on the transformed labels, and then make a prediction on the test set and undo the transformation with the inverse of the logit function (logistic function). Frustratingly though, this gives me even worse results than the naive linear regression!

Any suggestions on other transformations I can try or what I am doing wrong?

  • $\begingroup$ Logistic regression if the values are ratios of counts; if they're a mix of 0/1 and continuous you may need to do something else $\endgroup$ – Glen_b -Reinstate Monica Nov 28 '13 at 10:35
  • $\begingroup$ @Glen_b It's a mix of exactly 0 and 1's and continuous values between 0 and 1. If they were ratio of counts, couldn't there be exactly 0 and 1's as well? $\endgroup$ – mchangun Nov 28 '13 at 10:47
  • $\begingroup$ Yes, if they were ratios of counts they could certainly be 0 and 1 as well -- that presents no problem for logistic regression. However, it's not suitable for the continuous values between 0 and 1. The 0s and 1s may need to be dealt with somewhat differently from the values in between. $\endgroup$ – Glen_b -Reinstate Monica Nov 28 '13 at 16:19
  • $\begingroup$ What about a censored regression model? See e.g. cran.r-project.org/web/packages/censReg/vignettes/censReg.pdf $\endgroup$ – Fabian Nov 28 '13 at 21:41
  • $\begingroup$ Do you know how the ratios were computed ? That is, do you know numerator and denominator counts? Or can estimate them somehow? If so you can transform data in form for logistic regression. $\endgroup$ – kjetil b halvorsen Sep 6 '18 at 19:54

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Browse other questions tagged or ask your own question.