# References for weighted linear regression with linear constraints on the coefficients?

I would like to estimate a multiple linear regression model $N$ observations (with $\beta$ of length $k$):

$$Y = X \beta + \epsilon$$

subject, however, to some linear constraints on the coefficients. I.e a constraint of the form

$$M \beta = 0$$

where (at least in my case) $M$ is of dimension $2 \times k$ but more generally can be expressed as

$$M \beta = c$$ where $M$ is of dimension $p \times k$ with $p < N$ and $c$ of dimension $p \times 1$.

Thanks

-
It's not quite clear what your problem is here. What are these "other weights z"? Also, it sounds like your issue with the constraints on beta is actually the problem rather than anything to do with the weighting. So perhaps you should be a bit more specific about what those constraints are. –  Peter Ellis Jan 19 '12 at 18:19
I've edited my question...I basically want to know where I can read about/ learn about regression with these types of constraints. –  Palace Chan Jan 19 '12 at 18:56
I'm not the swiftest with mathematical notation...what is p? What is the significance of p being multiplied by 1? And is this a mixture design? –  rolando2 Jan 19 '12 at 19:46
@rolando2: Not a mixture model..and p isnt multiplied by 1..it's just px1 in dimensions...no particular significance im just saying some of the coefficients to be estimated must satisfy linear constraints..in the above notation $p$ just happens to be the number of such linear constriants. –  Palace Chan Jan 19 '12 at 22:25
Pretty much any linear regression theory textbook will provide the derivation and (closed-form) solution for finding the least-squares estimates of the coefficients under linear equality constraints. See, for example, G. A. F. Seber and A. J. Lee, Linear Regression Analysis, 2nd. ed., Sec. 3.8, pp 59-62. –  cardinal Jan 20 '12 at 2:27

The slides from the website below give a very detailed procedure on how to transform your equality constrained least squares into simply an unconstrained least squares problem. It uses QR decomposition to split the normal equations into a part that depends on the constraints and another part that does not. I hope this proves useful!

The slides you're looking for are on 19,20 & 21.

See: http://folk.uio.no/inf9540/CLS.pdf (warning: .pdf)

-
Thank you for the reference. Welcome to our site! –  whuber Jan 20 '12 at 14:34
For the case of weighted regression, we usually try to minimize $\displaystyle\sum_{i,j} ((Y_{i,j} - X\hat{B}_{i,j})w_{ij})^2$ by choosing some $\hat{B}$. That is, we compute the matrix of error terms, weigh them by w, and then determine which $\hat{B}$ minimizes the Frobenius norm.