Skip to main content

Questions tagged [constrained-regression]

constrained regression is (linear or nonlinear) regression models with further constraints on the coefficients. That could be non-negativity constraints, constraints on the norm of the coefficient vector or otherwise.

Filter by
Sorted by
Tagged with
2 votes
1 answer
54 views

Regression with known upper bounds and lower bounds of predicted variables

I have three variables $x_1$, $x_2$ and $x_3$ to predict $y$. Simplest regression setup is to run regression $y \sim x_1 + x_2 + x_3$. Then I have prediction $\hat{y} = \hat \beta_1 x_1 + \hat \...
Taylor Fang's user avatar
2 votes
0 answers
28 views

In the mgcv ::bam function in R, how can I constrain a two dimensional smooth to be monotonically increasing in both dimensions for large data?

I have a large dataset (1.3M rows) where I want to ensure that both Age and Duration increase monotonically for each by factor level (Male, Female). Here is the setup of the model: ...
Colin's user avatar
  • 41
0 votes
0 answers
18 views

Can we use dummy variables to capture the nonlinearity in Age, Period, and Cohort effects?

According to the chapterBayesian Age-Period-Cohort Models written by Ethan Fosse, the Classical APC Regression Model is the method I referred to in the question. In my specific case, I opt to set the ...
doraemon's user avatar
  • 250
2 votes
0 answers
62 views

Constrained least squares where at least one of two coefficients is zero

I have a linear model with a bunch of variables a number of linear constraints on these variables. I am currently using quadratic programming to solve this constrained least squares problem. However, ...
galpo's user avatar
  • 121
3 votes
1 answer
173 views

Bayesian linear regression: How to enforce constraint on the sum of coefficients?

I have a linear regression problem in which my $X$ matrix is not full rank. Here is a small example: $$X = \left[\begin{array}{rrrr} -1 & 0 & 0 & 1 \\ 1 & 0 & -1 & 0 \\ 0 &...
ischmidt20's user avatar
1 vote
0 answers
60 views

How to fit using a model, which has two highly correlated parameters? [closed]

I am trying to fit a dataset that depends on one observable, i.e. x. The model has two parameters that are highly correlated, of the form: $$f(x, \frac{\alpha}{\...
John's user avatar
  • 11
0 votes
0 answers
82 views

Constrained regression with multiple sum=0 constraints

this is my first post on this website so please advice me if I can add any relevant information. I'm running into a problem of how properly set up a model by using the Sum to zero constraint in Python....
Antonio Amoretti's user avatar
3 votes
2 answers
107 views

Gaussian Process with non-negative basis coefficients

I'm interested in adding basis functions to a Gaussian Process. In particular, following Section 2 of Rasmussen's book, I have $$g(x)=f(x) + h(x)^\top\beta,\qquad f(x)\sim\mathcal{GP}\left(0,k\left(x,...
foreignvol's user avatar
3 votes
2 answers
428 views

Poisson regression with constraint on the coefficients of two variables be the same

The aim of this experiment is to explore the effects of age, period and cohort. Thus, none of them can be thrown. Therefore, the assumption of no cohort effects greatly simplifies estimations but can ...
doraemon's user avatar
  • 250
0 votes
1 answer
101 views

Fitting a segmented regression with two zero boundary constraints

I am having trouble with this segmented regression as it requires two constraints and so far I have only treated single constraints. Here is an example of some data I am trying to fit: ...
leonardo's user avatar
4 votes
1 answer
325 views

How to constrain regression coefficients to be proportional

Given the regression model $Y = \beta_1 X_1 + \beta_2 X_2 + \epsilon$ I would like to constrain $\beta_2$ to be proportional to $\beta_1$, that is $\beta_2 = \theta\cdot\beta_1$. $Y = \beta_1 X_1 + \...
Reylined's user avatar
3 votes
2 answers
534 views

fitting a gam with constraints on parameters to deal with separation in parametric terms

I am trying to fit a gam using mgcv which has a mix of smooth and parametric terms. The model is for some count data on fish catches. I am modelling variation in location and time, but also ...
chris's user avatar
  • 41
1 vote
0 answers
36 views

Negative Binomial Regression with Non-Negative Coefficients

I am working with counts data and I need the estimated coefficients for the mean to be non-negative for interpretation reasons. So far, the only two packages support this are addreg and glmnet. ...
zhli12's user avatar
  • 21
1 vote
0 answers
58 views

Relaxed non-negative least squares

I am reconstructing a probability vector from data using non-negative least squares: $$ \sum_\alpha \left(\pi_\alpha - \sum_i W_{\alpha i}p_i\right)^2\rightarrow \min,\\ p_i\geq 0,\sum_i p_i=1 $$ ...
Roger V.'s user avatar
  • 4,429
0 votes
0 answers
98 views

Posterior covariance of the function values and derivatives of a Gaussian Process

I'm trying to figure out how to calculate the cross-terms of the covariance matrix between the function values and derivatives of a Gaussian Process. For context, this is needed for Gaussian Process ...
Eddy's user avatar
  • 101
1 vote
1 answer
351 views

How do I fit a constrained logistic regression model via quadratic programming in R?

I trying to find $\pi_{1}, \pi_{2}, \pi_{3}$ for model: $$ Y = \pi_{1}X_{1} + \pi_{2}X_{2} + \pi_{3}X_{3} + \epsilon, $$ with constraints: $\Sigma_{k}\pi_{k}=1$ and $\pi_{k}\geq0$. (All $\pi$ are ...
user366142's user avatar
1 vote
0 answers
445 views

Force selected coefficients to be non-negative in ridge regression

I want to fit a ridge regression on ~ 47 variables and 12 of them I want to be positive (or at least non-negative). I'm using sklearn and doing the following: ...
Chronicles's user avatar
1 vote
1 answer
80 views

Shrinkage methods: does non-convex constraints ($q < 1$) also induce sparsity?

I'm reading ESLII, in particular the chapter about shrinkage methods, lasso, and ridge regression. The optimal model parameters for a given constraint $\sum | \beta_j |^q < \alpha $ are given by ...
alexmolas's user avatar
  • 288
2 votes
0 answers
84 views

Confidence intervals for non-negative least squares

Can we use the non-parametric bootstrapping to compute the confidence intervals for the regression coefficients estimated from non-negative least squares? I wonder whether this has the same issues ...
shani's user avatar
  • 681
3 votes
1 answer
38 views

Allow gradient descent to go beyond constraint but punish for it

Disclaimer: I considered posting this on mathSE but thought maybe this is more fitting here (no pun intended). Status Quo I have a list of data points $\{x_k, y_k\}$ and a set of functions $f_i(x)$ ...
infinitezero's user avatar
1 vote
0 answers
31 views

Add constraints to linear regression which make some of the coefficients positive [duplicate]

I run a linear model which is about sales~ Investments, denote the Investments as I and ...
nobodyishere's user avatar
0 votes
1 answer
127 views

Testing for plateau with discrete timepoints

I have measured a continuous outcome at 4 timepoints (t1, t2, t3, t4), spaced evenly apart. The measurements at each time point are on different groups (i.e. the responses at t1 come from all ...
Jacob's user avatar
  • 41
6 votes
1 answer
553 views

Linear model with constraints in R

I have the following problem: I have a simple linear model with 3 independent variables. ...
victorhdd's user avatar
2 votes
1 answer
872 views

Fit points with a continuous, piecewise linear function with minimum number of points for each segment

I have a set of two-dimensional data points that I want to fit with a continuous piecewise linear function with one break point. However, I want each of the two segments to be supported by a minimum ...
frog's user avatar
  • 23
2 votes
0 answers
77 views

Covariance matrix of beta coefficients for constrained multiple regression

I have a linear least-squares problem with constraints that two of the coefficients must be non-negative. For a typical (unconstrained) least squares estimation, I know that the variance-covariance ...
cozisco's user avatar
  • 21
0 votes
0 answers
149 views

Why SCM weights *should* sum to 1?

When we use synthetic controls, we consider $j=2, \dots, J+1$ units across $t \in \{T_{-} \ldots T_0 \ldots T_{+}\} \in \mathbb{Z}$ pre/post event-time periods. Abadie et. al. make the point that to ...
Jared Greathouse's user avatar
0 votes
0 answers
60 views

How to Model Constrained Outcome in Multivariate Regression Context

I have a problem in which the ratio of two different outcomes as a function of time $\frac{Y_{1t}}{Y_{2t}}$ is unknown constant $c$. I would like to estimate regression models of the following type: $$...
ML0's user avatar
  • 1
1 vote
1 answer
187 views

How do linear constraints affect the convexity of my OLS-like optimisation problem?

I would like to augment a linear regression (so a convex OLS problem) with some additional constraints on the coefficients to match the subject I'm working on. Having $x\in \mathbb{R}^n$, the solution ...
quentin's user avatar
  • 11
1 vote
1 answer
227 views

Minimizing $L_2$ norm with constrained residual sum of squares (RSS)

I have some complex-valued time-series data, $y \in \mathbb{C}^n$ - a signal with additive Gaussian white noise. The goal is to find the Fourier coefficients of this signal. Ideally, you would just do ...
XYZT's user avatar
  • 213
0 votes
0 answers
175 views

How to get standard errors of linear regression coefficients subject to linear equality constraints

I have a regression problem with multiple sets of sum-to-one categorical features (e.g. X1=Male, X2=Female, X3=Weekday, X4=Weekend), intercept needs to be included as well as some restrictions on the ...
regression_practitioner's user avatar
0 votes
0 answers
179 views

Neural network regression with a constrained dependent variable

I'm using a multi-layer perceptron model to predict the cumulative number of boardings and alightings from a bus travelling on a standard route based on previous cumulative boarding and alighting ...
Sagnik Datta's user avatar
1 vote
0 answers
229 views

Difference between glmnet and nnls for non-negative least squares in R

I'm trying to do some non-negative linear regressions in R, and I found in the blog here https://www.r-bloggers.com/2019/11/non-negative-least-squares/ that either the package ...
zhli12's user avatar
  • 21
3 votes
2 answers
2k views

How to fit a constrained regression in R?

My regression: $Y = α + β_1X_1 + β_2X_2 + β_3X_3 + β_4X_4 + β_5X_5 + β_6X_6 + β_7X_7 + β_8X_8 + β_9X_9 + β_{10}X_{10} + β_{11}X_{11} + β_{12}X_{12} + β_{13}X_{13} + β_{14}X_{14} + ε$ Constraints that ...
JVaLi's user avatar
  • 31
1 vote
1 answer
689 views

Can it be proven that non-negative least squares is a perfectly convex problem?

Can it be proven that NNLS is a perfectly convex problem? Myre writes in section 3 of his TNT-NN manuscript: The NNLS objective function is a convex quadratic function with linear inequalities as ...
zdebruine's user avatar
  • 261
1 vote
1 answer
145 views

Numerical solution to the constrained ridge regression

The constrained ridge regression problem is of the form: $\arg\min_{\|\beta\|_2\le t}\|X\beta-y\|_2$. Given a matrix $X$, a vector $y$ and the constrain parameter $t$, how do you solve it numerically? ...
try's user avatar
  • 11
0 votes
0 answers
30 views

Constraining regression coefficient to non-negative [duplicate]

I have a regression problem where I don't want the coefficients to be negative. Is setting negative coefficients of OLS to zero the same as constraining the coefficient to be non-zero and solving it ...
stevew's user avatar
  • 831
2 votes
0 answers
51 views

How to get accurate estimates on Neural Networks Hessian?

I need to get not only accurate estimates on the neural network output itself but also on its second order derivatives in order to use the NN for optimization problems. With Adam optimizer I can't get ...
Saul Santos's user avatar
1 vote
0 answers
421 views

Linear Regression with constraint(s): need s.e., t-stat, and p-values of the regressors

I am performing a linear regression and what I need is (1) to constrain the sum of the regression coefficients to 1, and (2) to constrain the sum of regression coefficients to 1 AND each regression ...
Supratim Das Gupta's user avatar
0 votes
0 answers
51 views

Perform regression on a set of equations with additional constraints

I want to analyse the composition of a ternary mixture (a mixture containing of three components) by combining measurements of the mixture's density (denoted $\rho$) and its refractive index (denoted $...
albert's user avatar
  • 101
0 votes
0 answers
22 views

Can regression coefficients be guaranteed to be normalized?

When solving a multivariate linear regression problem of the form $A\vec{x}=\vec{y}$ where $A$ and $\vec{y}$ are known. Is there any from of preprocessing or scaling that can be done $A$ and or $\vec{...
ClimbingTheCurve's user avatar
3 votes
3 answers
4k views

How to test if sum of two coefficients of ols model is greater than zero using R?

The regression model is: y = b0 +b1x1 + b2x2 + b3x3 + e I want to test if b1 + b2 > 0. the R package ...
peter's user avatar
  • 33
2 votes
1 answer
82 views

How can this model be fit using R's lm function?

I have data that consists of a grouping variable $\tt{grp}$, a predictor $\tt{x}$, and a response $\tt{y}$. There are three groups. I want to fit to this data the model $y = \alpha_i + (\beta + \...
VKV's user avatar
  • 289
0 votes
1 answer
90 views

Constrained optimization of joint Bernoulli density parametrized by a neural network

I want to learn a joint distribution on $n$ Bernoulli random variables conditioned on a random variable $b\sim D$ and parametrized by a neural network, $f$: $$p(A = (a_1, \ldots, a_n)|b) = f_{\Lambda}(...
MRicci's user avatar
  • 151
4 votes
0 answers
65 views

Regression with coefficients having a multiplicative relationship [duplicate]

Assume I need to calibrate a linear model of the form shown here: $$Y\sim\beta_0+\beta_1X_1+\beta_2X_2$$ where the $\beta$ terms are the coefficients and the $X$ terms are the independent variables. ...
Max's user avatar
  • 206
1 vote
0 answers
32 views

Least-squares fit of explicit parabolic sheet to scattered data points [duplicate]

For a given set of data points $$\{(x_i, y_i, z_i)\}$$ there exists some $$f_{ABC}(x,y)=Ax^2+Bxy+Cy^2$$ that minimizes $$\sum_i(f_{ABC}(x_i,y_i)-z_i)^2$$ $A$, $B$, and $C$ can be found quickly ...
Museful's user avatar
  • 375
7 votes
2 answers
4k views

Monotonic splines in Python [closed]

I am trying to find a procedure to fit data monotonically in Python. The data won’t be necessarily monotonic. I just would like to achieve a monotonic fit because of theoretical assumptions. I ...
Eaglez's user avatar
  • 81
4 votes
1 answer
1k views

How do you apply constrains on parameters in Bayesian modeling?

In Frequentist modeling, I know how to fit and interpret models with constraints on the parameter space. Let's say I'm fitting a $N(\mu, \sigma^2)$ distribution to my data $x_1, \dots, x_n$, and I ...
Eli's user avatar
  • 2,672
4 votes
0 answers
77 views

Linear regression of higher order polynomial with slope constraint

I am trying to constrain the coefficients on a higher order polynomial (let's say an order 6) for the curve to be decreasing. I have found this link, where the fitting of a 3rd order polynomial is ...
Stata_user's user avatar
3 votes
1 answer
97 views

How to fit several linear models with linear intercept-slope constrain?

I have a data where several groups exist: $i = 1, 2, 3...n$ ($n$ about 600 groups in total). For group $i$ theory predicts a linear relation $y = a_i + b_i * x$ with $a_i$ and $b_i$ intercept and ...
Bibe's user avatar
  • 33
2 votes
2 answers
1k views

Adding limits to regression coefficients

For my problem, I have data that contains daily observations of the total time and the volumes of task A completed, task B completed, C, D.. and I am looking to estimate the time it takes to solve ...
Youbloodywombat's user avatar