# Elastic net/LASSO with soft labels

Sometimes you do not have firm Y/N labels, but e.g. 80% probability of Y as a label. E.g. this happens, if you train a model on a small amount of labelled data, predict for a large amount of unlabelled data and then want to use the predictions as soft-labels.

If I want to use soft labels, then a lot of software for elastic net logistic regression (e.g. glmnet in R) does not allow labels in (0,1). Ideas for software that can handle such soft labels directly is also welcome. One potential solution that occurred to me is to use observation weights - e.g. instead of a 0.8 soft label use create two observations with weights 0.2 and 0.8 and outcome 0 and 1, respectively.

Am I assuming correctly that that would work fine?

Any other special considerations (I could only think of perhaps making sure that such pairs of observations always end top in the same fold when doing cross-validation and to certainly not split such a pair across training and test data)?

• Beta regression? Through gamboostLSS potentially? Jun 19, 2019 at 17:53

Having said that, if I understood your problem correctly, I would recommend something like a mixture model. Indeed, you could write the likelihood of your model as: $$p(x, y | \theta) = p(x, y = 1 | \theta) p(y = 1) + p(x, y = 0 | \theta)p(y=0)$$ Where $$y$$ is the label, $$p(y=1) = 1 - p(y=0)$$ the soft label for $$y=1$$, $$x$$ the features and $$\theta$$ the parameters. $$p(x, y = 1 | \theta)$$ and $$p(x, y = 0 | \theta)$$ are the likelihoods of the logistic regression when $$y=1$$ or $$y=0$$ respectively.
If you are feeling Bayesian, you could do this using the probabilistic programming language Stan for example. The Lasso could be implemented using a Laplace distribution for the prior $$p(\theta)$$. For Elastic Net, you would use a mixture of Gaussian and Laplace distribution. Alternatively, you could also use something like a horseshoe prior.