# Questions tagged [constrained-regression]

constrained regression is (linear or nonlinear) regression models with further constraints on the coefficients. That could be non-negativity constraints, constraints on the norm of the coefficient vector or otherwise.

123 questions
Filter by
Sorted by
Tagged with
26 views

### Shrinkage methods: does non-convex constraints ($q < 1$) also induce sparsity?

I'm reading ESLII, in particular the chapter about shrinkage methods, lasso, and ridge regression. The optimal model parameters for a given constraint $\sum | \beta_j |^q < \alpha$ are given by ...
• 155
18 views

### Confidence intervals for non-negative least squares

Can we use the non-parametric bootstrapping to compute the confidence intervals for the regression coefficients estimated from non-negative least squares? I wonder whether this has the same issues ...
• 601
13 views

### Recover Primal Linear Programming Solution from Dual with LAD Regression?

This link discusses different ways of writing a classic LAD regression with a linear program. The classic way of writing LAD regression ($y = X \beta + r$) as a linear program is \...
23 views

### Allow gradient descent to go beyond constraint but punish for it

Disclaimer: I considered posting this on mathSE but thought maybe this is more fitting here (no pun intended). Status Quo I have a list of data points $\{x_k, y_k\}$ and a set of functions $f_i(x)$ ...
• 133
1 vote
27 views

### Add constraints to linear regression which make some of the coefficients positive [duplicate]

I run a linear model which is about sales~ Investments, denote the Investments as I and ...
18 views

### Testing for plateau with discrete timepoints

I have measured a continuous outcome at 4 timepoints (t1, t2, t3, t4), spaced evenly apart. The measurements at each time point are on different groups (i.e. the responses at t1 come from all ...
79 views

### Linear model with constraints in R

I have the following problem: I have a simple linear model with 3 independent variables. ...
1 vote
123 views

### Fit points with a continuous, piecewise linear function with minimum number of points for each segment

I have a set of two-dimensional data points that I want to fit with a continuous piecewise linear function with one break point. However, I want each of the two segments to be supported by a minimum ...
• 13
28 views

### Covariance matrix of beta coefficients for constrained multiple regression

I have a linear least-squares problem with constraints that two of the coefficients must be non-negative. For a typical (unconstrained) least squares estimation, I know that the variance-covariance ...
• 21
43 views

### Why SCM weights *should* sum to 1?

When we use synthetic controls, we consider $j=2, \dots, J+1$ units across $t \in \{T_{-} \ldots T_0 \ldots T_{+}\} \in \mathbb{Z}$ pre/post event-time periods. Abadie et. al. make the point that to ...
36 views

• 151
36 views

### Regression with coefficients having a multiplicative relationship

Assume I need to calibrate a linear model of the form shown here: $$Y\sim\beta_0+\beta_1X_1+\beta_2X_2$$ where the $\beta$ terms are the coefficients and the $X$ terms are the independent variables. ...
• 176
1 vote
29 views

### Least-squares fit of explicit parabolic sheet to scattered data points [duplicate]

For a given set of data points $$\{(x_i, y_i, z_i)\}$$ there exists some $$f_{ABC}(x,y)=Ax^2+Bxy+Cy^2$$ that minimizes $$\sum_i(f_{ABC}(x_i,y_i)-z_i)^2$$ $A$, $B$, and $C$ can be found quickly ...
• 365
2k views

### Monotonic splines in Python [closed]

I am trying to find a procedure to fit data monotonically in Python. The data won’t be necessarily monotonic. I just would like to achieve a monotonic fit because of theoretical assumptions. I ...
• 71
393 views

### How do you apply constrains on parameters in Bayesian modeling?

In Frequentist modeling, I know how to fit and interpret models with constraints on the parameter space. Let's say I'm fitting a $N(\mu, \sigma^2)$ distribution to my data $x_1, \dots, x_n$, and I ...
• 2,089
65 views

### Linear regression of higher order polynomial with slope constraint

I am trying to constrain the coefficients on a higher order polynomial (let's say an order 6) for the curve to be decreasing. I have found this link, where the fitting of a 3rd order polynomial is ...
• 231
69 views

### How to fit several linear models with linear intercept-slope constrain?

I have a data where several groups exist: $i = 1, 2, 3...n$ ($n$ about 600 groups in total). For group $i$ theory predicts a linear relation $y = a_i + b_i * x$ with $a_i$ and $b_i$ intercept and ...
• 33
575 views

### Adding limits to regression coefficients

For my problem, I have data that contains daily observations of the total time and the volumes of task A completed, task B completed, C, D.. and I am looking to estimate the time it takes to solve ...
119 views

1k views

### Predict a vector of values with constraints? [duplicate]

I am aware of a variety of methods for simultaneously predicting multiple outcomes known sometimes as multivariate regression/analysis. However, my situation is a little more special. I am trying to ...
• 441
320 views

### How can you constrain values to be positive when fitting a model?

I'm currently fitting a model using maximum likelihood estimation on biological data (electroencephalography). Basically, I'm fitting normal distributions to several subsets of data (experimental ...
418 views

### Equality-constrained least-squares when the matrix is singular

I have to solve the linear system $Ax = b$ in the least-square sense, where matrix $A$ is singular. To resolve this, I introduce $Cx = d$. However, even after this, I am not able to solve it using <...
• 151
2k views

### Is it possible to apply a monotonicity constraint on a Gaussian process regression fit?

Below is a code using scikit-learn where I simply apply Gaussian process regression (GPR) on a set of observed data to produce an expected fit. I know physically that this curve should be ...
• 447
1 vote
701 views

### R - Linear Regression with parameters constraint - contr.sum contrast

I am facing a problem with running a simple OLS regression with two categorical independent variable. I would like to impose that the sum of the coefficients referred to one variable (x in the ...
105 views

### How to run linear regression with constraints in R? [duplicate]

If I have the following data n<-1000 x1<-rnorm(n,1,1) x2<-rnorm(n,2,2) x3<-rnorm(n,3,3) e<-rnorm(n) y<-3+0.5*x1+0.2*x2+0.3*x3+e I want to fit a ...
• 181
1k views

### method for predicting a curve

I have data on several curves. the data is of the form: curve_id x y and there are many x/y pairs for each curve and x is limited to some known range. overall, the curves look quite similar in ...
• 551
575 views

### polynomial regression in R: how to add hard constraints (go through specified points)? [duplicate]

I'd like to perform polynomial regression on my data (which lies in [0,10]), but I need to ensure that the endpoints of the range are fixed, i.e. that the curve goes through (0,0) and (10,10). So the ...
1 vote
294 views

### nlme ignoring certain control arguments to nlminb

Issue: Do constrained optimization of parameters in nlme::nlme I'm trying to fit a non-linear mixed effects model using nlme::nlme, which can use 2 optimization schemes: stats::nlminb or stats::nlm. ...
• 31
77 views

### Confidence limits for constrained penalized log likelihood model

I am estimating parameter $\beta$ as: \begin{align} \hat \beta &= \mathop{\mathrm{arg\,max}}_\beta \;\; l(\beta;X,y) - \frac{\lambda}{2}\left(\tilde y-g(\beta,\tilde X)\right)^\prime C^\prime C\...
• 113
Is nonnegative least squares already a form of regularisation? By adding a constraint that $\beta \geq 0$ (the coefficients), does it make sense to add another regularisation term as in LASSO or ridge ...