How to fit this linear regression with constraints? I would like to fit the following model:
$$Y=\beta_0+\beta_1(\sum_{i=1}^kw_iX_i)+\beta_2(\sum_{i=1}^kw_iX_i)^2+\epsilon$$
where $\beta_0, \beta_1, \beta_2, w_1,...,w_k$ are the parameters, and $\epsilon$ is some normal noise. It does not look like something I have encountered before, and I know it's different from simply including all the second order and interaction terms, since the coefficients are related/fixed in a specific way through the sharing of the weights $w_i$'s. It seems that this is some sort of linear regression with constraints that relate the coefficients. Could someone point me in the right direction how such model can be fitted?
 A: The model is overparametrised: you don't need $\beta_1$, which can be set to anything convenient, like 1. 
One thing I thought of was to fit iteratively.  Start out with some guess at $w$ and $\beta_2$. Then compute $Z=(\sum_i \hat{w}_iX_i)^2$ and fit the linear model
Y~ X1+X2+...+X_k + Z
The coefficients of the $X$s are the new $\hat{w}_i$, and the coefficient of $Z$ is $\hat \beta_2$.  And then recompute Z,  iterate and hope it converges. Sadly, it doesn't.
But if  $k$ isn't too large, it's easy to just compute the residual sum of squares as a function of the parameters and run it through a general purpose optimiser. In R I'd use minqa::newuoa, but there are lots of alternatives.
> X<-matrix(rnorm(50*100),ncol=5)
> w<-1:5
> Y<- (X%*%w)+2*(X%*%w)^2+rnorm(100)
> 
> 
> rss<-function(theta){
+   beta2<-theta[1]
+   w<-theta[-1]
+   mu<- (X%*%w)+beta2*(X%*%w)^2
+   sum((Y-mu)^2)
+ }
> 
> minqa::newuoa(par=rep(1,6), rss)
parameter estimates: 1.99478699135839, 1.00032043499982, 2.00140284432351, 3.00312315850919, 4.00284240744153, 5.00537517104468 
objective: 1047.51402563294 
number of function evaluations: 1689 

Then use the bootstrap to get standard error estimates.
With $k=50$ it doesn't work (without tuning -- I'm sure it would work if the optimiser defaults were changed or the starting values were better)
