# Least Squares with coefficient constraints

I need to perform a least squares problem with constraints on some (but not all) of my coefficients.

For example say I am fitting the following model:

$$\hat{y} = \beta_0 + \beta_1 x_1 + \beta_2 x_2 + \beta_3 x_3 + \beta_4 x_4$$

where I need

$$\beta_2 \geq 0$$

and

$$\beta_3 \leq 0$$

and all other coefficients left unconstrained.

If my constraint was that $\beta_i \geq 0$ for all $i$ then I know my problem could be solved as:

$$\arg\min_{x \geq 0} \left(\frac{1}{2}x^T Q x + c^Tx\right)$$m

with

$$Q = A^T A$$ and $$c = -A^T y$$

which can be solved with scipy.optimize.nnls. I'm certain my modification isn't that exotic, and there's already software that does it - I just can't seem to put it in the correct form.

• Are you looking for a closed-form solution (which AFAIK doesn't exist), or a specific Python function (which does exist: scipy.optimize.least_squares, where you let some parameters float freely by including np.inf with an appropriate sign in the bounds parameter)? – Stephan Kolassa Jun 18 '18 at 10:11
• Ideally a closed form, or an algorithm. – Tom Kealy Jun 18 '18 at 10:14
• Let $x_3^*=-x_3$, which flips the sign of the negative beta and use a constrained LS routine that lets you specify non-negative coefficients, then at the end, negate the resulting estimate of $\beta_3$ – Glen_b Jun 18 '18 at 12:25