I need to perform a least squares problem with constraints on some (but not all) of my coefficients.

For example say I am fitting the following model:

$$\hat{y} = \beta_0 + \beta_1 x_1 + \beta_2 x_2 + \beta_3 x_3 + \beta_4 x_4 $$

where I need

$$ \beta_2 \geq 0 $$


$$ \beta_3 \leq 0 $$

and all other coefficients left unconstrained.

If my constraint was that $ \beta_i \geq 0 $ for all $ i $ then I know my problem could be solved as:

$$ \arg\min_{x \geq 0} \left(\frac{1}{2}x^T Q x + c^Tx\right) $$m


$$ Q = A^T A $$ and $$ c = -A^T y $$

which can be solved with scipy.optimize.nnls. I'm certain my modification isn't that exotic, and there's already software that does it - I just can't seem to put it in the correct form.

  • 2
    $\begingroup$ Are you looking for a closed-form solution (which AFAIK doesn't exist), or a specific Python function (which does exist: scipy.optimize.least_squares, where you let some parameters float freely by including np.inf with an appropriate sign in the bounds parameter)? $\endgroup$ – Stephan Kolassa Jun 18 '18 at 10:11
  • $\begingroup$ Ideally a closed form, or an algorithm. $\endgroup$ – Tom Kealy Jun 18 '18 at 10:14
  • 2
    $\begingroup$ Let $x_3^*=-x_3$, which flips the sign of the negative beta and use a constrained LS routine that lets you specify non-negative coefficients, then at the end, negate the resulting estimate of $\beta_3$ $\endgroup$ – Glen_b Jun 18 '18 at 12:25

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Browse other questions tagged or ask your own question.