Questions tagged [constrained-regression]

constrained regression is (linear or nonlinear) regression models with further constraints on the coefficients. That could be non-negativity constraints, constraints on the norm of the coefficient vector or otherwise.

Filter by
Sorted by
Tagged with
2 votes
0 answers
54 views

Constrained least squares where at least one of two coefficients is zero

I have a linear model with a bunch of variables a number of linear constraints on these variables. I am currently using quadratic programming to solve this constrained least squares problem. However, ...
galpo's user avatar
  • 21
3 votes
1 answer
110 views

Bayesian linear regression: How to enforce constraint on the sum of coefficients?

I have a linear regression problem in which my $X$ matrix is not full rank. Here is a small example: $$X = \left[\begin{array}{rrrr} -1 & 0 & 0 & 1 \\ 1 & 0 & -1 & 0 \\ 0 &...
ischmidt20's user avatar
1 vote
0 answers
54 views

How to fit using a model, which has two highly correlated parameters? [closed]

I am trying to fit a dataset that depends on one observable, i.e. x. The model has two parameters that are highly correlated, of the form: $$f(x, \frac{\alpha}{\...
John's user avatar
  • 11
0 votes
0 answers
64 views

Constrained regression with multiple sum=0 constraints

this is my first post on this website so please advice me if I can add any relevant information. I'm running into a problem of how properly set up a model by using the Sum to zero constraint in Python....
Antonio Amoretti's user avatar
0 votes
0 answers
29 views

Scale dependent solutions for scale-invariant cost function in nonlinear regression

Suppose, I have a nonlinear regression problem, where I have the lables of my training data $\bf{y}$ and two measurement matrixes $A$ and $B$. The cost function is $\Vert \frac{A*\textbf{w}}{B*\textbf{...
baronett's user avatar
0 votes
0 answers
75 views

Multivariate regression with constrained target

Consider the multivariate regression $Y = (\hat{y}_1, \hat{y}_2) = f(x_1,\ldots,x_n, y_1/y_2) = f(X)$; as we see, the ratio of outputs $\frac{y_1}{y_2}$ is among the inputs $X$, thus a trained model ...
krzysiekb's user avatar
2 votes
2 answers
374 views

Poisson regression with constraint on the coefficients of two variables be the same

The aim of this experiment is to explore the effects of age, period and cohort. Thus, none of them can be thrown. Therefore, the assumption of no cohort effects greatly simplifies estimations but can ...
doraemon's user avatar
  • 198
0 votes
1 answer
88 views

Fitting a segmented regression with two zero boundary constraints

I am having trouble with this segmented regression as it requires two constraints and so far I have only treated single constraints. Here is an example of some data I am trying to fit: ...
leonardo's user avatar
4 votes
1 answer
320 views

How to constrain regression coefficients to be proportional

Given the regression model $Y = \beta_1 X_1 + \beta_2 X_2 + \epsilon$ I would like to constrain $\beta_2$ to be proportional to $\beta_1$, that is $\beta_2 = \theta\cdot\beta_1$. $Y = \beta_1 X_1 + \...
Reylined's user avatar
3 votes
2 answers
424 views

fitting a gam with constraints on parameters to deal with separation in parametric terms

I am trying to fit a gam using mgcv which has a mix of smooth and parametric terms. The model is for some count data on fish catches. I am modelling variation in location and time, but also ...
chris's user avatar
  • 41
1 vote
0 answers
31 views

Negative Binomial Regression with Non-Negative Coefficients

I am working with counts data and I need the estimated coefficients for the mean to be non-negative for interpretation reasons. So far, the only two packages support this are addreg and glmnet. ...
zhli12's user avatar
  • 21
1 vote
0 answers
54 views

Relaxed non-negative least squares

I am reconstructing a probability vector from data using non-negative least squares: $$ \sum_\alpha \left(\pi_\alpha - \sum_i W_{\alpha i}p_i\right)^2\rightarrow \min,\\ p_i\geq 0,\sum_i p_i=1 $$ ...
Roger V.'s user avatar
  • 3,743
0 votes
0 answers
88 views

Posterior covariance of the function values and derivatives of a Gaussian Process

I'm trying to figure out how to calculate the cross-terms of the covariance matrix between the function values and derivatives of a Gaussian Process. For context, this is needed for Gaussian Process ...
Eddy's user avatar
  • 101
1 vote
1 answer
320 views

How do I fit a constrained logistic regression model via quadratic programming in R?

I trying to find $\pi_{1}, \pi_{2}, \pi_{3}$ for model: $$ Y = \pi_{1}X_{1} + \pi_{2}X_{2} + \pi_{3}X_{3} + \epsilon, $$ with constraints: $\Sigma_{k}\pi_{k}=1$ and $\pi_{k}\geq0$. (All $\pi$ are ...
user366142's user avatar
1 vote
0 answers
371 views

Force selected coefficients to be non-negative in ridge regression

I want to fit a ridge regression on ~ 47 variables and 12 of them I want to be positive (or at least non-negative). I'm using sklearn and doing the following: ...
Chronicles's user avatar
1 vote
1 answer
68 views

Shrinkage methods: does non-convex constraints ($q < 1$) also induce sparsity?

I'm reading ESLII, in particular the chapter about shrinkage methods, lasso, and ridge regression. The optimal model parameters for a given constraint $\sum | \beta_j |^q < \alpha $ are given by ...
alexmolas's user avatar
  • 286
2 votes
0 answers
77 views

Confidence intervals for non-negative least squares

Can we use the non-parametric bootstrapping to compute the confidence intervals for the regression coefficients estimated from non-negative least squares? I wonder whether this has the same issues ...
shani's user avatar
  • 671
3 votes
1 answer
36 views

Allow gradient descent to go beyond constraint but punish for it

Disclaimer: I considered posting this on mathSE but thought maybe this is more fitting here (no pun intended). Status Quo I have a list of data points $\{x_k, y_k\}$ and a set of functions $f_i(x)$ ...
infinitezero's user avatar
1 vote
0 answers
31 views

Add constraints to linear regression which make some of the coefficients positive [duplicate]

I run a linear model which is about sales~ Investments, denote the Investments as I and ...
nobodyishere's user avatar
0 votes
1 answer
81 views

Testing for plateau with discrete timepoints

I have measured a continuous outcome at 4 timepoints (t1, t2, t3, t4), spaced evenly apart. The measurements at each time point are on different groups (i.e. the responses at t1 come from all ...
Jacob's user avatar
  • 41
4 votes
1 answer
395 views

Linear model with constraints in R

I have the following problem: I have a simple linear model with 3 independent variables. ...
victorhdd's user avatar
2 votes
1 answer
631 views

Fit points with a continuous, piecewise linear function with minimum number of points for each segment

I have a set of two-dimensional data points that I want to fit with a continuous piecewise linear function with one break point. However, I want each of the two segments to be supported by a minimum ...
frog's user avatar
  • 23
2 votes
0 answers
76 views

Covariance matrix of beta coefficients for constrained multiple regression

I have a linear least-squares problem with constraints that two of the coefficients must be non-negative. For a typical (unconstrained) least squares estimation, I know that the variance-covariance ...
cozisco's user avatar
  • 21
0 votes
0 answers
138 views

Why SCM weights *should* sum to 1?

When we use synthetic controls, we consider $j=2, \dots, J+1$ units across $t \in \{T_{-} \ldots T_0 \ldots T_{+}\} \in \mathbb{Z}$ pre/post event-time periods. Abadie et. al. make the point that to ...
Jared Greathouse's user avatar
0 votes
0 answers
58 views

How to Model Constrained Outcome in Multivariate Regression Context

I have a problem in which the ratio of two different outcomes as a function of time $\frac{Y_{1t}}{Y_{2t}}$ is unknown constant $c$. I would like to estimate regression models of the following type: $$...
ML0's user avatar
  • 1
1 vote
1 answer
185 views

How do linear constraints affect the convexity of my OLS-like optimisation problem?

I would like to augment a linear regression (so a convex OLS problem) with some additional constraints on the coefficients to match the subject I'm working on. Having $x\in \mathbb{R}^n$, the solution ...
quentin's user avatar
  • 11
1 vote
1 answer
198 views

Minimizing $L_2$ norm with constrained residual sum of squares (RSS)

I have some complex-valued time-series data, $y \in \mathbb{C}^n$ - a signal with additive Gaussian white noise. The goal is to find the Fourier coefficients of this signal. Ideally, you would just do ...
XYZT's user avatar
  • 213
0 votes
0 answers
164 views

How to get standard errors of linear regression coefficients subject to linear equality constraints

I have a regression problem with multiple sets of sum-to-one categorical features (e.g. X1=Male, X2=Female, X3=Weekday, X4=Weekend), intercept needs to be included as well as some restrictions on the ...
regression_practitioner's user avatar
0 votes
0 answers
167 views

Neural network regression with a constrained dependent variable

I'm using a multi-layer perceptron model to predict the cumulative number of boardings and alightings from a bus travelling on a standard route based on previous cumulative boarding and alighting ...
Sagnik Datta's user avatar
1 vote
0 answers
188 views

Difference between glmnet and nnls for non-negative least squares in R

I'm trying to do some non-negative linear regressions in R, and I found in the blog here https://www.r-bloggers.com/2019/11/non-negative-least-squares/ that either the package ...
zhli12's user avatar
  • 21
3 votes
2 answers
1k views

How to fit a constrained regression in R?

My regression: $Y = α + β_1X_1 + β_2X_2 + β_3X_3 + β_4X_4 + β_5X_5 + β_6X_6 + β_7X_7 + β_8X_8 + β_9X_9 + β_{10}X_{10} + β_{11}X_{11} + β_{12}X_{12} + β_{13}X_{13} + β_{14}X_{14} + ε$ Constraints that ...
JVaLi's user avatar
  • 31
0 votes
1 answer
622 views

Can it be proven that non-negative least squares is a perfectly convex problem?

Can it be proven that NNLS is a perfectly convex problem? Myre writes in section 3 of his TNT-NN manuscript: The NNLS objective function is a convex quadratic function with linear inequalities as ...
zdebruine's user avatar
  • 241
1 vote
1 answer
129 views

Numerical solution to the constrained ridge regression

The constrained ridge regression problem is of the form: $\arg\min_{\|\beta\|_2\le t}\|X\beta-y\|_2$. Given a matrix $X$, a vector $y$ and the constrain parameter $t$, how do you solve it numerically? ...
try's user avatar
  • 11
0 votes
0 answers
30 views

Constraining regression coefficient to non-negative [duplicate]

I have a regression problem where I don't want the coefficients to be negative. Is setting negative coefficients of OLS to zero the same as constraining the coefficient to be non-zero and solving it ...
stevew's user avatar
  • 831
2 votes
0 answers
50 views

How to get accurate estimates on Neural Networks Hessian?

I need to get not only accurate estimates on the neural network output itself but also on its second order derivatives in order to use the NN for optimization problems. With Adam optimizer I can't get ...
Saul Santos's user avatar
1 vote
0 answers
397 views

Linear Regression with constraint(s): need s.e., t-stat, and p-values of the regressors

I am performing a linear regression and what I need is (1) to constrain the sum of the regression coefficients to 1, and (2) to constrain the sum of regression coefficients to 1 AND each regression ...
Supratim Das Gupta's user avatar
0 votes
0 answers
50 views

Perform regression on a set of equations with additional constraints

I want to analyse the composition of a ternary mixture (a mixture containing of three components) by combining measurements of the mixture's density (denoted $\rho$) and its refractive index (denoted $...
albert's user avatar
  • 101
0 votes
0 answers
22 views

Can regression coefficients be guaranteed to be normalized?

When solving a multivariate linear regression problem of the form $A\vec{x}=\vec{y}$ where $A$ and $\vec{y}$ are known. Is there any from of preprocessing or scaling that can be done $A$ and or $\vec{...
ClimbingTheCurve's user avatar
3 votes
3 answers
3k views

How to test if sum of two coefficients of ols model is greater than zero using R?

The regression model is: y = b0 +b1x1 + b2x2 + b3x3 + e I want to test if b1 + b2 > 0. the R package ...
peter's user avatar
  • 33
2 votes
1 answer
78 views

How can this model be fit using R's lm function?

I have data that consists of a grouping variable $\tt{grp}$, a predictor $\tt{x}$, and a response $\tt{y}$. There are three groups. I want to fit to this data the model $y = \alpha_i + (\beta + \...
VKV's user avatar
  • 289
0 votes
1 answer
87 views

Constrained optimization of joint Bernoulli density parametrized by a neural network

I want to learn a joint distribution on $n$ Bernoulli random variables conditioned on a random variable $b\sim D$ and parametrized by a neural network, $f$: $$p(A = (a_1, \ldots, a_n)|b) = f_{\Lambda}(...
MRicci's user avatar
  • 151
3 votes
0 answers
52 views

Regression with coefficients having a multiplicative relationship

Assume I need to calibrate a linear model of the form shown here: $$Y\sim\beta_0+\beta_1X_1+\beta_2X_2$$ where the $\beta$ terms are the coefficients and the $X$ terms are the independent variables. ...
Max's user avatar
  • 176
1 vote
0 answers
31 views

Least-squares fit of explicit parabolic sheet to scattered data points [duplicate]

For a given set of data points $$\{(x_i, y_i, z_i)\}$$ there exists some $$f_{ABC}(x,y)=Ax^2+Bxy+Cy^2$$ that minimizes $$\sum_i(f_{ABC}(x_i,y_i)-z_i)^2$$ $A$, $B$, and $C$ can be found quickly ...
Museful's user avatar
  • 375
6 votes
2 answers
4k views

Monotonic splines in Python [closed]

I am trying to find a procedure to fit data monotonically in Python. The data won’t be necessarily monotonic. I just would like to achieve a monotonic fit because of theoretical assumptions. I ...
Eaglez's user avatar
  • 71
4 votes
1 answer
970 views

How do you apply constrains on parameters in Bayesian modeling?

In Frequentist modeling, I know how to fit and interpret models with constraints on the parameter space. Let's say I'm fitting a $N(\mu, \sigma^2)$ distribution to my data $x_1, \dots, x_n$, and I ...
Eli's user avatar
  • 2,642
4 votes
0 answers
75 views

Linear regression of higher order polynomial with slope constraint

I am trying to constrain the coefficients on a higher order polynomial (let's say an order 6) for the curve to be decreasing. I have found this link, where the fitting of a 3rd order polynomial is ...
Stata_user's user avatar
3 votes
1 answer
97 views

How to fit several linear models with linear intercept-slope constrain?

I have a data where several groups exist: $i = 1, 2, 3...n$ ($n$ about 600 groups in total). For group $i$ theory predicts a linear relation $y = a_i + b_i * x$ with $a_i$ and $b_i$ intercept and ...
Bibe's user avatar
  • 33
2 votes
2 answers
914 views

Adding limits to regression coefficients

For my problem, I have data that contains daily observations of the total time and the volumes of task A completed, task B completed, C, D.. and I am looking to estimate the time it takes to solve ...
Youbloodywombat's user avatar
0 votes
0 answers
150 views

Is there a conjugate prior for normal location family N(x|u,1) such that the mean is always positive?

The conjugate prior to normal location family is usually a normal distribution. However, I want to constrain the mean to be positive. Is there a conjugate prior to the normal location family $x\sim N(...
user31575's user avatar
3 votes
1 answer
201 views

Model/link function to deal with dependent variable in range [-1,1]?

My dependent variable, $Y$, contains values anywhere from -1 to 1 (i.e. it is bounded continuously on the range $[-1,1]$). I know that a regular OLS regression on such a variable would sometimes ...
Coolio2654's user avatar