Questions tagged [constrained-regression]

constrained regression is (linear or nonlinear) regression models with further constraints on the coefficients. That could be non-negativity constraints, constraints on the norm of the coefficient vector or otherwise.

Filter by
Sorted by
Tagged with
0
votes
0answers
16 views

Can regression coefficients be guaranteed to be normalized?

When solving a multivariate linear regression problem of the form $A\vec{x}=\vec{y}$ where $A$ and $\vec{y}$ are known. Is there any from of preprocessing or scaling that can be done $A$ and or $\vec{...
0
votes
0answers
16 views

How to handle compositional explanatory features in redundancy analysis?

I have a matrix where rows are sites and columns are categories of land usage. The cell values add to one across each site row; that is, for each site I calculated land usage within a set radius. I ...
0
votes
0answers
16 views

“Out-of-sample updating”: Applying estimated rates of change from one sample to another

[If you know the terms I could use to find the literature on this, that information alone would be a very helpful answer.] My model uses a survey -- let's call it O, for "old" -- that ran ...
1
vote
3answers
56 views

How to test if sum of two coefficients of ols model is greater than zero using R?

The regression model is: y = b0 +b1x1 + b2x2 + b3x3 + e I want to test if b1 + b2 > 0. the R package ...
2
votes
1answer
34 views

How can this model be fit using R's lm function?

I have data that consists of a grouping variable $\tt{grp}$, a predictor $\tt{x}$, and a response $\tt{y}$. There are three groups. I want to fit to this data the model $y = \alpha_i + (\beta + \...
0
votes
1answer
36 views

Constrained optimization of joint Bernoulli density parametrized by a neural network

I want to learn a joint distribution on $n$ Bernoulli random variables conditioned on a random variable $b\sim D$ and parametrized by a neural network, $f$: $$p(A = (a_1, \ldots, a_n)|b) = f_{\Lambda}(...
3
votes
0answers
21 views

Regression with coefficients having a multiplicative relationship

Assume I need to calibrate a linear model of the form shown here: $$Y\sim\beta_0+\beta_1X_1+\beta_2X_2$$ where the $\beta$ terms are the coefficients and the $X$ terms are the independent variables. ...
0
votes
0answers
16 views

Least squares fit of a bivariate quadratic-linear product to an oriented point set

As explained in this question, a bivariate quadratic has 6 DoF (coefficients), and a bivariate cubic has 10 DoF, while a bivariate quadratic-linear product has 8 DoF. The quadratic or the cubic models ...
1
vote
0answers
28 views

Least-squares fit of explicit parabolic sheet to scattered data points [duplicate]

For a given set of data points $$\{(x_i, y_i, z_i)\}$$ there exists some $$f_{ABC}(x,y)=Ax^2+Bxy+Cy^2$$ that minimizes $$\sum_i(f_{ABC}(x_i,y_i)-z_i)^2$$ $A$, $B$, and $C$ can be found quickly ...
2
votes
1answer
93 views

Monotonic splines in Python [closed]

I am trying to find a procedure to fit data monotonically in Python. The data won’t be necessarily monotonic. I just would like to achieve a monotonic fit because of theoretical assumptions. I ...
0
votes
0answers
19 views

In constrained optimization, what does projecting the gradient into the tangent space of the feasible region mean?

While going through the book "DeepLearning.org", section 4.3 (Constrained Optmization), it is stated that : If we use a small constant step size e(epselon), we can make gradient descent steps, then ...
3
votes
1answer
46 views

How do you apply constrains on parameters in Bayesian modeling?

In Frequentist modeling, I know how to fit and interpret models with constraints on the parameter space. Let's say I'm fitting a $N(\mu, \sigma^2)$ distribution to my data $x_1, \dots, x_n$, and I ...
0
votes
0answers
22 views

How to estimate non-negative least squares with weight constraints

I'm using scipy.optimize.nnls to estimate Non-Negative Least Squares. I want to add a constraint in the estimation that no single feature exceeds 20% of the total ...
0
votes
0answers
47 views

Can lsqlincon (or a similar function) in R replicate results of lsqlin in MATLAB for data matrices that are not full column rank?

I am trying to replicate code written originally in MATLAB / Octave, in the R programming language. This is a constrained optimization problem using the lsqlin function available in MATLAB and Octave. ...
3
votes
0answers
42 views

Linear regression of higher order polynomial with slope constraint

I am trying to constrain the coefficients on a higher order polynomial (let's say an order 6) for the curve to be decreasing. I have found this link, where the fitting of a 3rd order polynomial is ...
0
votes
0answers
40 views

Inequality Constrained Least Squares: Computation of the Confidence Interval of the mean response

Suppose I wish to find perform inequality constraint least squares on the model $$ y=X\beta +\epsilon$$ subject to a set of constraints $$ T\beta>c \space \space.$$ Is there a way to compute the ...
3
votes
1answer
59 views

How to fit several linear models with linear intercept-slope constrain?

I have a data where several groups exist: $i = 1, 2, 3...n$ ($n$ about 600 groups in total). For group $i$ theory predicts a linear relation $y = a_i + b_i * x$ with $a_i$ and $b_i$ intercept and ...
0
votes
0answers
81 views

Relation between regularization and Lagrange multipliers

This is sort of a follow-up to What is the connection between regularization and the method of lagrange multipliers ? that I thought deserved a new question. Consider an optimization problem of the ...
0
votes
0answers
63 views

Constrained parameters update during hidden Markov model forward-backward algorithm

I want to train a Gaussian hidden Markov model. I currently use the Python package hmmlearn. I looked thouroghly at the code to see how parameters are updated after each training iteration, but I ...
1
vote
1answer
88 views

Adding limits to regression coefficients

For my problem, I have data that contains daily observations of the total time and the volumes of task A completed, task B completed, C, D.. and I am looking to estimate the time it takes to solve ...
0
votes
0answers
52 views

Is there a conjugate prior for normal location family N(x|u,1) such that the mean is always positive?

The conjugate prior to normal location family is usually a normal distribution. However, I want to constrain the mean to be positive. Is there a conjugate prior to the normal location family $x\sim N(...
3
votes
1answer
34 views

Model/link function to deal with dependent variable in range [-1,1]?

My dependent variable, $Y$, contains values anywhere from -1 to 1 (i.e. it is bounded continuously on the range $[-1,1]$). I know that a regular OLS regression on such a variable would sometimes ...
1
vote
0answers
45 views

Sum of residuals in constrained linear regression

Does sum of residuals == 0 hold even if we add ordinality constraints for the coefficients in a regression model? It would be great if someone can point out to any literature (paper/presentation/...
0
votes
0answers
50 views

Closed-form solutions for constrained multiple linear regression

Normally a multiple linear regression is unconstrained $$y=X\beta+\epsilon$$ so that closed-form solutions in the case of data orthogonality ($X^\top X=I$) are $$\beta=(X^\top X)^{-1} X^\top y$$ ...
0
votes
0answers
142 views

Restricted Weighted Linear Regression in R

I have to follwing issue. I would like to run a linear regression imposing a constraint on the weighted coefficients. Let me construct an example: Consider the following cross-sectional regression $...
2
votes
2answers
317 views

Predict a vector of values with constraints? [duplicate]

I am aware of a variety of methods for simultaneously predicting multiple outcomes known sometimes as multivariate regression/analysis. However, my situation is a little more special. I am trying to ...
0
votes
1answer
47 views

How can you constrain values to be positive when fitting a model?

I'm currently fitting a model using maximum likelihood estimation on biological data (electroencephalography). Basically, I'm fitting normal distributions to several subsets of data (experimental ...
0
votes
1answer
70 views

least square with equality constraint and singular matrix

I have the following problem to solve: Ax = b where A is singular. To resolve this I introduce a condition Cx = d Even after this, I am not able to solve it using scipy.optimize.minimize as it ...
0
votes
1answer
869 views

Is it possible to apply a monotonicity constraint on a Gaussian process regression fit?

Below is a code using scikit-learn where I simply apply Gaussian process regression (GPR) on a set of observed data to produce an expected fit. I know physically that this curve should be ...
1
vote
0answers
337 views

R - Linear Regression with parameters constraint - contr.sum contrast

I am facing a problem with running a simple OLS regression with two categorical independent variable. I would like to impose that the sum of the coefficients referred to one variable (x in the ...
3
votes
0answers
70 views

How to run linear regression with constraints in R? [duplicate]

If I have the following data n<-1000 x1<-rnorm(n,1,1) x2<-rnorm(n,2,2) x3<-rnorm(n,3,3) e<-rnorm(n) y<-3+0.5*x1+0.2*x2+0.3*x3+e I want to fit a ...
1
vote
1answer
485 views

method for predicting a curve

I have data on several curves. the data is of the form: curve_id x y and there are many x/y pairs for each curve and x is limited to some known range. overall, the curves look quite similar in ...
0
votes
1answer
295 views

polynomial regression in R: how to add hard constraints (go through specified points)? [duplicate]

I'd like to perform polynomial regression on my data (which lies in [0,10]), but I need to ensure that the endpoints of the range are fixed, i.e. that the curve goes through (0,0) and (10,10). So the ...
1
vote
0answers
128 views

nlme ignoring certain control arguments to nlminb

Issue: Do constrained optimization of parameters in nlme::nlme I'm trying to fit a non-linear mixed effects model using nlme::nlme, which can use 2 optimization schemes: stats::nlminb or stats::nlm. ...
0
votes
1answer
72 views

Confidence limits for constrained penalized log likelihood model

I am estimating parameter $\beta$ as: \begin{align} \hat \beta &= \mathop{\mathrm{arg\,max}}_\beta \;\; l(\beta;X,y) - \frac{\lambda}{2}\left(\tilde y-g(\beta,\tilde X)\right)^\prime C^\prime C\...
3
votes
1answer
234 views

Is constrained (nonnegative) least squares a form a regularisation?

Is nonnegative least squares already a form of regularisation? By adding a constraint that $\beta \geq 0$ (the coefficients), does it make sense to add another regularisation term as in LASSO or ridge ...
10
votes
3answers
1k views

Calculating the p-values in a constrained (non-negative) least squares

I have been using Matlab to perform unconstrained least squares (ordinary least squares) and it automatically outputs the coefficients, test statistic and the p-values. My question is, upon ...
1
vote
1answer
800 views

Constrained Linear Regression with coefficients related by inequality

I have a (slightly simplified) model of the following form: $Y=c_1X_1 + c_2X_2 + \epsilon$ subject to the constraint $0\leq c_1\leq c_2$. The distribution of $\epsilon$ is actually not important to ...
0
votes
1answer
353 views

RDA and CCA output- unconstrained inertia 0.00 rank 0

I get an ouput for RDA and for CCA that says that my unconstrained inertia is 0, rank 0. I thought that would be a good thing in the meaning that all the variance in the data is explained by my (...
1
vote
0answers
268 views

Comparing log-transformed and non-log transformed models

Suppose I have the following two models fitted with constrained linear least squares: Model 1: $Y = \beta_1X_1 + \ldots + \beta_kX_k + \varepsilon$ Model 2: $\log_{10}(Y) = \beta_1\log_{10}(X_1) + \...
3
votes
1answer
119 views

How to perform Least Squares with constraints on a subset of the model coefficients?

For solving an unconstrained LS regression $$\hat{y}=w_1.x_1+w_2.x_2+w_3.x_3+w_4.x_4 + \epsilon$$ I use the following normal equation: $$W^*=(X^{\top}X)^{-1}X^{\top}Y $$ If I want to introduce a ...
23
votes
2answers
1k views

The limit of “unit-variance” ridge regression estimator when $\lambda\to\infty$

Consider ridge regression with an additional constraint requiring that $\hat{\mathbf y}$ has unit sum of squares (equivalently, unit variance); if needed, one can assume that $\mathbf y$ has unit sum ...
1
vote
0answers
70 views

How to create a constrained spline meta model in R?

Background I have an outcome measure I'm trying to predict that is on a scale 0-1000. I've trained (say) M different models (model definition doesn't matter) on overlapping sub-ranges of the data (e....
5
votes
1answer
1k views

What are drawbacks of isotonic regression?

I have been reading about isotonic regression and it seems like a great method that will give one a monotone regression function estimator and, moreover, is free of any tuning parameters. Why are ...
3
votes
0answers
182 views

Fit big GAM with constraints on linear terms

I'm trying to fit MGCV GAM with constraints on linear terms. I feed a dataset with factor variable G. Number of rows is about 30-40k. Number of model parameters - ...
1
vote
0answers
81 views

Choice of ridge parameter in constrained estimation

Apologies for a possible basic question. Suppose I have a model $$y=X\beta + \epsilon, \quad E[\epsilon|X]=0, \quad X \text{ is } n \times k$$ and, say, matrix $X'X$ is nearly singular and I have a ...
1
vote
1answer
139 views

Fitting a regression with constraints on the relationship between one independent variable and the dependent variable?

Let's say I have a dependent variable y (a rating of the pleasantness of shopping at a particular store from 0 to 100), and 10 independent variables ...
0
votes
1answer
117 views

Estimation with adding-up constraints when the constraint is a coefficient

I have a variable $Y_i = Y_{1,i} + Y_{2,i} + \cdots + Y_{n,i}$. Specifically, $Y_i$ is a measure of consumption for an individual $i$ and $Y_{1,i},Y_{2,i},\cdots$ are consumption expenditures on ...
1
vote
0answers
133 views

R: Constrained coefficients for fitting a linear model to a subset of the data

I fit a regression with 37 variables to my entire dataset and got regression results. One of the variable is the distance in miles (dist). I believe that my data follows two distinct regimes with ...
0
votes
1answer
56 views

Optimization with changing constraints based on parameters to optimize?

I have a large dataset consisting of an ordered categorical variable that is broken into exclusive binary vectors, one for each level of the category. I would like to find a linear combination of ...