I need to perform a least squares problem with constraints on some (but not all) of my coefficients.

For example say I am fitting the following model:

$$\hat{y} = \beta_0 + \beta_1 x_1 + \beta_2 x_2 + \beta_3 x_3 + \beta_4 x_4 $$

where I need

$$ \beta_2 \geq 0 $$


$$ \beta_3 \leq 0 $$

and all other coefficients left unconstrained.

If my constraint was that $ \beta_i \geq 0 $ for all $ i $ then I know my problem could be solved as:

$$ \arg\min_{x \geq 0} \left(\frac{1}{2}x^T Q x + c^Tx\right) $$m


$$ Q = A^T A $$ and $$ c = -A^T y $$

which can be solved with scipy.optimize.nnls. I'm certain my modification isn't that exotic, and there's already software that does it - I just can't seem to put it in the correct form.

  • 2
    $\begingroup$ Are you looking for a closed-form solution (which AFAIK doesn't exist), or a specific Python function (which does exist: scipy.optimize.least_squares, where you let some parameters float freely by including np.inf with an appropriate sign in the bounds parameter)? $\endgroup$ Jun 18, 2018 at 10:11
  • $\begingroup$ Ideally a closed form, or an algorithm. $\endgroup$
    – Tom Kealy
    Jun 18, 2018 at 10:14
  • 2
    $\begingroup$ Let $x_3^*=-x_3$, which flips the sign of the negative beta and use a constrained LS routine that lets you specify non-negative coefficients, then at the end, negate the resulting estimate of $\beta_3$ $\endgroup$
    – Glen_b
    Jun 18, 2018 at 12:25


Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Browse other questions tagged or ask your own question.