Linear regression with upper and/or lower limits in R?

Is there a way to run a linear regression with upper and/or lower limits on the coefficients in R?

-

Yes, you can do this in Lavaan.

Here's an example. We fit a regression model, and find an estimate of 0.10. Then fit a model using lavaan, and get the same parameter estimate. Then fit the model with a constraint that b1 has to be greater than 0.

library(lavaan)
set.seed(1234)
df <- as.data.frame(matrix(rnorm(500), ncol=2))
names(df) <- c("x", "y")
summary(glm(y ~ x, data=df))

lavModel1 <- 'y ~ b1*x'

summary(sem(lavModel1, df))

lavModel2 <- 'y ~ b1*x
b1 > 0'
summary(sem(lavModel2, df))

-
"Greater than zero [or whatever other constant or other model coefficient one wants to use as a lower bound]". I think this works for upper bounds too, no? Not sure if you can have both bounds on a single coefficient, but I bet you can. –  Nick Stauner May 3 at 20:27
Yes, you can have any set of constraints you like. You can have b1 < sqrt(variance(x)) and b1 > (b2 * b3). (Not that you'd want to.) –  Jeremy Miles May 3 at 20:31

Yes, you can do it by re-defining linear regression as an optimization problem (a residual sum of squares cost function for example) and solving it using the constraints you want. So to use the numbers as the example from @Jeremy: (Note that in this example we are fitting a model without an intercept)

set.seed(1234)
df <- as.data.frame(matrix(rnorm(500), ncol=2))
names(df) <- c("x", "y")
CostFunction <- function(theta){sum( (df$y - theta*df$x)^2)}
theta_0 =1;
theta_opt <- optim(fn= CostFunction, lower=0, par = theta_0, method="L-BFGS-B")


Ultimately any package you wish to use for constrained regression will do the same thing: formulate a cost function and solve a constrained optimization problem. In the case of ordinary least squares as the one shown here "ordinary" optimizers using "simple" BFGS variants will do just fine; harder problems (eg. GLMM) will require more exotic beasts like BOBYQA.

-