2
$\begingroup$

I have three variables $x_1$, $x_2$ and $x_3$ to predict $y$. Simplest regression setup is to run regression $y \sim x_1 + x_2 + x_3$. Then I have prediction $\hat{y} = \hat \beta_1 x_1 + \hat \beta_2 x_2 + \hat \beta_3 x_3 $

But now, for each observation $y_i$, I actually know $l_i \leq y_i \leq u_i$. So theoretically I can set my prediction to be $\hat{y} = \max( \min( u_i, \hat \beta_1 x_1 + \hat \beta_2 x_2 + \hat \beta_3 x_3 ), l_i )$. But in this case I am running a regression without using the knowledge of $l_i \leq y_i \leq u_i$, and I simply clip $\hat y$ after I obtain the regression coefficients. However I do want to take advantage of this $l_i, u_i$ information in my "training" process. How do I solve this problem?

I can't simply do something like $\min || y - (\beta_1 x_1 + \beta_2 x_2 + \beta_3 x_3) ||^2 $ subject to $l_i \leq (\beta_1 x_{i1} + \beta_2 x_{i2} + \beta_3 x_{i3}) \leq u_i $. We might not be able to guarantee I can find $\beta_1, \beta_2, \beta_3$ that satisfy all constraints. And my predictor doesn't require me to find $\beta_1, \beta_2, \beta_3$ that satisfy all constraints. As mentioned I can just cap $\hat{y} = \max( \min( u_i, \hat \beta_1 x_1 + \hat \beta_2 x_2 + \hat \beta_3 x_3 ), l_i )$ but I want to utilize $l_i, u_i$ information in my "training" process

$\endgroup$

1 Answer 1

1
$\begingroup$

It sounds like what you want is tobit regression, which is designed for data exactly like yours, where any predicted values above an upper bounds are set to that upper bound and values below a lower bound are set to that lower bound. That is, tobit regression posits the following latent variable model: $$ Y^*_i = X_i\beta + \varepsilon_i $$ with $Y_i=u_i$ if $Y_i^* \ge u_i$, $Y_i=l_i$ if $Y_i^* \le l_i$, and $Y_i=Y_i^*$ otherwise, where $Y^*$ is a latent (unobserved) variable and $Y$ is the observed outcome. This model can be fit using maximum likelihood assuming $\varepsilon_i \sim \mathcal{N}(0, \sigma^2)$.

Variations exist with $u_i = \infty$ or $l_i = -\infty$ (i.e., one-sided tobit models). $u_i$ and $l_i$ are provided by the user and not estimated. The coefficients are interpreted just as they are for a linear model, except they refer to the effect of the predictor on the latent variable. Tobit regression can be fit using standard software.

$\endgroup$
2
  • $\begingroup$ Thanks Noah. It does sound exactly what I am looking for. However I don't see how it's implemented based on en.wikipedia.org/wiki/Tobit_model Do you have a reference on how it's implemented? I also don't see a "standard" python implementation in popular libraries .... $\endgroup$ Commented Jun 10 at 1:14
  • $\begingroup$ That article gives the log likelihood for the one-sided tobit, which can pretty easily extended to the two-sided, and manually maximized. I also found this implementation in Python with a quick google search: github.com/jamesdj/tobit $\endgroup$
    – Noah
    Commented Jun 10 at 1:58

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.