# Constrained Regression in R: coefficients positive, sum to 1 and non-zero intercept

I have the model that I need to estimate, $$Y = \pi_0 + \pi_1 X_1 + \pi_2 X_2 + \pi_3 X_3 + \varepsilon,$$ with $\sum_k \pi_k = 1 \text{ for }k \geq 1$ and $\pi_k\ge0 \text{ for }k \geq 1$.

Elvis answer to another question solves this for the case of $\pi_0 = 0$. Here's his/her code of this solution:

   > library("quadprog");
> X <- matrix(runif(300), ncol=3)
> Y <- X %*% c(0.2,0.3,0.5) + rnorm(100, sd=0.2)
> Rinv <- solve(chol(t(X) %*% X));
> C <- cbind(rep(1,3), diag(3))
> b <- c(1,rep(0,3))
> d <- t(Y) %*% X
> solve.QP(Dmat = Rinv, factorized = TRUE, dvec = d, Amat = C, bvec = b, meq = 1)
$solution  0.2049587 0.3098867 0.4851546$value
 -16.0402

$unconstrained.solution  0.2295507 0.3217405 0.5002459$iterations
 2 0

$Lagrangian  1.454517 0.000000 0.000000 0.000000$iact
 1


How can I adjust this code such that it can estimate an intercept?

This has been cross-posted here because my group in my assignment is getting annoyed that I haven't estimated this regression yet. I will answer this question here if/when the other forum participants get there first.

You just need to play around a little with the matrices involved. Add the intercept to X:

XX <- cbind(1,X)


Recalculate the D matrix used in solve.QP() (I prefer working directly with this to avoid calling solve():

Dmat <- t(XX)%*%XX


Recalculate d with the new XX:

dd <- t(Y)%*%XX


Change the constraint matrix by adding a zero column, since you seem to not have any constraints on the intercept (right?):

Amat <- t(cbind(0,rbind(1,diag(3))))


And finally:

solve.QP(Dmat = Dmat, factorized = FALSE, dvec = dd, Amat = Amat, bvec = b, meq = 1)

• Thanks Stephan. I want to use a bootstrap to estimate the standard errors of these coefficient estimates. I will use a serial correlation and hetero robust bootstrap. Do you have any opinions on the legitimacy of this approach (the bootstrap in general, not the exact type of bootstrap that I'm going to use)?
– user14281
Oct 25 '12 at 8:40
• In principle, it looks like the bootstrap is legitimate here... as long as you keep in mind the constraints under which it was calculated and communicate these constraints clearly, but you were going to do that, weren't you ;-)? But I'm far from an expert on this; perhaps someone more competent can comment? Oct 25 '12 at 8:48
• I would be cautious about the bootstrap if any of the estimated $\pi_i$, $i \ge 1$, are zero, because that solution lies along a boundary of the parameter space and the regularity conditions used to justify bootstrapping might not hold. But that's precisely the case where this solution is needed: after all, if all the $\pi_i$ are nonzero, then one could just use OLS to solve this problem.
– whuber
Oct 25 '12 at 10:34
• @whuber: good point. Of course, the other goal of this thing is that the coefficients not only be nonnegative, but also sum to 1. Suppose all coefficients are well away from 0. Would the bootstrap with the sum constraint be valid? Oct 25 '12 at 12:26
• The sum restricts the parameters to a submanifold of the original (one without boundaries or corners). Thus it does not have the same effect as imposing a boundary condition and should not affect estimation, bootstrapping, etc. Another way to see this would be to reduce the number of parameters by one, say by rewriting $\pi_3 = 1 - (\pi_1+\pi_2)$ and rewriting the non-negativity conditions as inequalities in $\pi_1$ and $\pi_2$ only. (Conditions to force all values to be less than or equal to $1$ are then unnecessary.)
– whuber
Oct 25 '12 at 13:14