4
$\begingroup$

I am trying to do a LASSO regression on some data. However, my dependent variable is between 0 and 1. How do I go about this? Do I just apply a sigmoid function to the regression output?

This will surely force the outcome to the 0-1 range, but I am not sure of the technical implications.

$\endgroup$
6
  • $\begingroup$ Do you mean your dependent (output) variable? $\endgroup$
    – Kevin
    Jul 28, 2017 at 14:46
  • 2
    $\begingroup$ how would you model this data if you weren't interested in penalization during estimation? Answering this question should send you in the right direction. $\endgroup$
    – user795305
    Jul 28, 2017 at 14:49
  • $\begingroup$ I am not sure either. So even if I was to do a straight linear regression, my question would still stand. $\endgroup$
    – Minaj
    Jul 28, 2017 at 14:58
  • 2
    $\begingroup$ If $x$ can take any value but $y$ is bounded between 0 and 1, then $y$ isn't a linear function of $x$. You've specified two properties of the function (that $y \in (0,1)$, and something about it being continuous). There are all kinds of crazy looking nonlinear functions that satisfy these properties. To get to the point of fitting a model, you'd have to be more explicit about the type of function you're looking for. $\endgroup$
    – user20160
    Jul 28, 2017 at 15:35
  • $\begingroup$ Is it a continuous proportion or a count proportion you are modelling? $\endgroup$
    – usεr11852
    Jul 29, 2017 at 8:04

3 Answers 3

5
$\begingroup$

Since the response variable is between 0 to 1, i.e., you should perform a beta regression. The package 'gamlss' allows you to do that in addition to fit your model using Lasso.

library(betareg)
data(GasolineYield)
library(gamlss)

X <- with(GasolineYield, cbind(gravity,pressure,temp10,temp,batch))
# standarise data 1-------------------------------------------------------------
sX <- scale(X)
# ridge
m1 <- gamlss(yield~ri(sX), data = GasolineYield)
# lasso
m2 <- gamlss(yield~ri(sX, Lp=1), data = GasolineYield)
# best subset
m3 <- gamlss(yield~ri(sX, Lp=0), data = GasolineYield)

# summary
summary(m1)
summary(m2)
summary(m3)

# plotting the coefficients
plot(getSmo(m1))
plot(getSmo(m2))
plot(getSmo(m3))

There are some variations for beta regression. Take a look at the GAMLSS Manual.

$\endgroup$
2
  • $\begingroup$ +1, and is my approach valid? $\endgroup$
    – Haitao Du
    Jul 28, 2017 at 15:38
  • $\begingroup$ If you apply a logit transformation (your sigmoid function if I understood correctly) then fit a linear model, Ferrari and Cribari-Neto (2004) states that you would make your residuals asymmetric. $\endgroup$ Jul 28, 2017 at 15:46
2
$\begingroup$

I am not sure, but I think we can do

$$ \text{minimize}~ \|\frac 1 {1+e^{-X\beta}} -y \|_2^2+ \lambda\|\beta\|_1 $$

Where $X$ is the data matrix and $y$ is the response and $\beta$ is the coefficients. The objective is convex.

And

$$ 0< \frac 1 {1+e^{-X\beta}} < 1$$

$\endgroup$
0
0
$\begingroup$

Let's say that the true relationship between predictors and response is (mostly) linear. In this case, you could do a regression and then truncate the outputs (i.e. anything below 0 counts as 0, anything above 1 counts as 1). This would be better than applying the sigmoid function.

If you used a sigmoid function, you'd want to do so while training the model (not simply applying it to a linear regression output); this would be better if your problem is closer to classification (i.e., most of your outputs are near 0 or near 1). (The betareg package manual mentions this idea too).

Ultimately, you'd want to use a plot of the data or some knowledge about its structure to make a final decision (per @user20160's comment).

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.