Logistic regression loss with continuous labels

Question

I want to learn a logistic regression model but my outcome variable can take continuous values in [0;1], not only binary labels {0;1}.

My model thus still needs a logistic link (to bound my predictions in [0;1]) but the log loss is not appropriate in my case. I cannot formulate my problem as a series of Bernoulli trials as in standard binary LR.

Any suggestion for n appropriate loss in my case?

Answer 1

If $Y$ cannot be exactly 0 or 1, you can just take the logit transformation of $Y$ and use ordinary least squares. Such a model reasonably thinks of errors as being on the logit scale (scale of the linear predictor in the linear ols model). You cannot use nonlinear least squares because that would assume errors on the original $Y$ scale, resulting in predictions outside $[0,1]$. I'm not sure what your reference to log loss means. The transformed ols model would be assuming Gaussian errors.

Answer 2

This situation might be handled by what is called beta regression. It strictly only deals with outcomes over (0,1), but there is a useful practical transformation described on page 3 of the linked document if you need to cover [0,1]. There is an associated R package. This issue is discussed in a bit more detail on this Cross Validated page.

Answer 3

The logistic regression model by default outputs the probability of the input belonging to a given class. I assume that in order to use this model for your data you are going to interpret this class probability as the predicted value.

If so you can simply use the MSE as is commonly used as a loss function for regression problems.

Answer 4

Either:

• change loss type (e.g. use square loss and cap or rescale predictions)
• trick the model into thinking there is a $\alpha \in [0,1]$ probability of a given example being positive or negative. E.g. you have a sample with target $.7$, then duplicate the sample, one with target $1$ and weight of $.7$ and one with target $0$ and weight $.3$