Take the 2-minute tour ×
Cross Validated is a question and answer site for people interested in statistics, machine learning, data analysis, data mining, and data visualization. It's 100% free, no registration required.

Is there a way to run a linear regression with upper and/or lower limits on the coefficients in R?

share|improve this question

2 Answers 2

up vote 4 down vote accepted

Yes, you can do this in Lavaan.

Here's an example. We fit a regression model, and find an estimate of 0.10. Then fit a model using lavaan, and get the same parameter estimate. Then fit the model with a constraint that b1 has to be greater than 0.

library(lavaan)
set.seed(1234)
df <- as.data.frame(matrix(rnorm(500), ncol=2))
names(df) <- c("x", "y")
summary(glm(y ~ x, data=df))

lavModel1 <- 'y ~ b1*x'

summary(sem(lavModel1, df))

lavModel2 <- 'y ~ b1*x
  b1 > 0'
summary(sem(lavModel2, df))
share|improve this answer
    
"Greater than zero [or whatever other constant or other model coefficient one wants to use as a lower bound]". I think this works for upper bounds too, no? Not sure if you can have both bounds on a single coefficient, but I bet you can. –  Nick Stauner May 3 at 20:27
1  
Yes, you can have any set of constraints you like. You can have b1 < sqrt(variance(x)) and b1 > (b2 * b3). (Not that you'd want to.) –  Jeremy Miles May 3 at 20:31

Yes, you can do it by re-defining linear regression as an optimization problem (a residual sum of squares cost function for example) and solving it using the constraints you want. So to use the numbers as the example from @Jeremy: (Note that in this example we are fitting a model without an intercept)

set.seed(1234)
df <- as.data.frame(matrix(rnorm(500), ncol=2))
names(df) <- c("x", "y")
CostFunction <- function(theta){sum( (df$y - theta*df$x)^2)}
theta_0 =1; 
theta_opt <- optim(fn= CostFunction, lower=0, par = theta_0, method="L-BFGS-B")

Ultimately any package you wish to use for constrained regression will do the same thing: formulate a cost function and solve a constrained optimization problem. In the case of ordinary least squares as the one shown here "ordinary" optimizers using "simple" BFGS variants will do just fine; harder problems (eg. GLMM) will require more exotic beasts like BOBYQA.

share|improve this answer

Your Answer

 
discard

By posting your answer, you agree to the privacy policy and terms of service.

Not the answer you're looking for? Browse other questions tagged or ask your own question.