Questions tagged [constrained-regression]
constrained regression is (linear or nonlinear) regression models with further constraints on the coefficients. That could be non-negativity constraints, constraints on the norm of the coefficient vector or otherwise.
2
votes
2answers
71 views
Predict a vector of values with constraints? [duplicate]
I am aware of a variety of methods for simultaneously predicting multiple outcomes known sometimes as multivariate regression/analysis. However, my situation is a little more special. I am trying to ...
0
votes
1answer
42 views
How can you constrain values to be positive when fitting a model?
I'm currently fitting a model using maximum likelihood estimation on biological data (electroencephalography). Basically, I'm fitting normal distributions to several subsets of data (experimental ...
0
votes
1answer
23 views
least square with equality constraint and singular matrix
I have the following problem to solve:
Ax = b where A is singular.
To resolve this I introduce a condition Cx = d
Even after this, I am not able to solve it using scipy.optimize.minimize as it ...
0
votes
0answers
11 views
Multilevel modeling, constraint for positive values
I'm currently trying to fit a shifted inverse gaussian to reaction time data (always postive). My paramterization of the model includes 3 parameters, alpha, gamma and tau, which must always be ...
0
votes
1answer
123 views
Is it possible to apply a monotonicity constraint on a Gaussian process regression fit?
Below is a code using scikit-learn where I simply apply Gaussian process regression (GPR) on a set of observed data to produce an expected fit. I know physically that this curve should be ...
1
vote
0answers
97 views
R - Linear Regression with parameters constraint - contr.sum contrast
I am facing a problem with running a simple OLS regression with two categorical independent variable.
I would like to impose that the sum of the coefficients referred to one variable (x in the ...
3
votes
0answers
46 views
How to run linear regression with constraints in R? [duplicate]
If I have the following data
n<-1000
x1<-rnorm(n,1,1)
x2<-rnorm(n,2,2)
x3<-rnorm(n,3,3)
e<-rnorm(n)
y<-3+0.5*x1+0.2*x2+0.3*x3+e
I want to fit a ...
1
vote
1answer
47 views
method for predicting a curve
I have data on several curves. the data is of the form:
curve_id x y
and there are many x/y pairs for each curve and x is limited to some known range. overall, the curves look quite similar in ...
0
votes
1answer
87 views
polynomial regression in R: how to add hard constraints (go through specified points)? [duplicate]
I'd like to perform polynomial regression on my data (which lies in [0,10]), but I need to ensure that the endpoints of the range are fixed, i.e. that the curve goes through (0,0) and (10,10). So the ...
1
vote
0answers
58 views
nlme ignoring certain control arguments to nlminb
Issue: Do constrained optimization of parameters in nlme::nlme
I'm trying to fit a non-linear mixed effects model using nlme::nlme, which can use 2 optimization schemes: stats::nlminb or stats::nlm.
...
0
votes
1answer
61 views
Confidence limits for constrained penalized log likelihood model
I am estimating parameter $\beta$ as:
\begin{align}
\hat \beta &= \mathop{\mathrm{arg\,max}}_\beta \;\; l(\beta;X,y) - \frac{\lambda}{2}\left(\tilde y-g(\beta,\tilde X)\right)^\prime C^\prime C\...
2
votes
1answer
63 views
Is constrained (nonnegative) least squares a form a regularisation?
Is nonnegative least squares already a form of regularisation? By adding a constraint that $\beta \geq 0$ (the coefficients), does it make sense to add another regularisation term as in LASSO or ridge ...
0
votes
0answers
59 views
Unable to solve using lagrangian multipliers
Suppose
$$K(x,z) = \theta(x)^T \theta(z) = \left\{ \begin{array}{ll}
1 & \text{if } x = z \\
0 & \text{otherwise} \end{array} \right.
$$
and $y_1=+1$ or $-1$.
I ...
9
votes
3answers
326 views
Calculating the p-values in a constrained (non-negative) least squares
I have been using Matlab to perform unconstrained least squares (ordinary least squares) and it automatically outputs the coefficients, test statistic and the p-values.
My question is, upon ...
1
vote
1answer
356 views
Constrained Linear Regression with coefficients related by inequality
I have a (slightly simplified) model of the following form:
$Y=c_1X_1 + c_2X_2 + \epsilon$ subject to the constraint $0\leq c_1\leq c_2$.
The distribution of $\epsilon$ is actually not important to ...
0
votes
1answer
150 views
RDA and CCA output- unconstrained inertia 0.00 rank 0
I get an ouput for RDA and for CCA that says that my unconstrained inertia is 0, rank 0. I thought that would be a good thing in the meaning that all the variance in the data is explained by my (...
1
vote
0answers
87 views
Comparing log-transformed and non-log transformed models
Suppose I have the following two models fitted with constrained linear least squares:
Model 1: $Y = \beta_1X_1 + \ldots + \beta_kX_k + \varepsilon$
Model 2: $\log_{10}(Y) = \beta_1\log_{10}(X_1) + \...
3
votes
1answer
31 views
How to perform Least Squares with constraints on a subset of the model coefficients?
For solving an unconstrained LS regression
$$\hat{y}=w_1.x_1+w_2.x_2+w_3.x_3+w_4.x_4 + \epsilon$$
I use the following normal equation:
$$W^*=(X^{\top}X)^{-1}X^{\top}Y $$
If I want to introduce a ...
22
votes
2answers
791 views
The limit of “unit-variance” ridge regression estimator when $\lambda\to\infty$
Consider ridge regression with an additional constraint requiring that $\hat{\mathbf y}$ has unit sum of squares (equivalently, unit variance); if needed, one can assume that $\mathbf y$ has unit sum ...
1
vote
0answers
54 views
How to create a constrained spline meta model in R?
Background
I have an outcome measure I'm trying to predict that is on a scale 0-1000. I've trained (say) M different models (model definition doesn't matter) on overlapping sub-ranges of the data (e....
5
votes
1answer
409 views
What are drawbacks of isotonic regression?
I have been reading about isotonic regression and it seems like a great method that will give one a monotone regression function estimator and, moreover, is free of any tuning parameters.
Why are ...
2
votes
0answers
109 views
Fit big GAM with constraints on linear terms
I'm trying to fit MGCV GAM with constraints on linear terms. I feed a dataset with factor variable G. Number of rows is about 30-40k. Number of model parameters - ...
1
vote
0answers
72 views
Choice of ridge parameter in constrained estimation
Apologies for a possible basic question. Suppose I have a model
$$y=X\beta + \epsilon, \quad E[\epsilon|X]=0, \quad X \text{ is } n \times k$$
and, say, matrix $X'X$ is nearly singular and I have a ...
1
vote
1answer
115 views
Fitting a regression with constraints on the relationship between one independent variable and the dependent variable?
Let's say I have a dependent variable y (a rating of the pleasantness of shopping at a particular store from 0 to 100), and 10 independent variables ...
0
votes
1answer
74 views
Estimation with adding-up constraints when the constraint is a coefficient
I have a variable $Y_i = Y_{1,i} + Y_{2,i} + \cdots + Y_{n,i}$. Specifically, $Y_i$ is a measure of consumption for an individual $i$ and $Y_{1,i},Y_{2,i},\cdots$ are consumption expenditures on ...
1
vote
0answers
115 views
R: Constrained coefficients for fitting a linear model to a subset of the data
I fit a regression with 37 variables to my entire dataset and got regression results. One of the variable is the distance in miles (dist). I believe that my data follows two distinct regimes with ...
0
votes
1answer
44 views
Optimization with changing constraints based on parameters to optimize?
I have a large dataset consisting of an ordered categorical variable that is broken into exclusive binary vectors, one for each level of the category.
I would like to find a linear combination of ...
4
votes
1answer
302 views
ridge regression parameter space
In Ridge regression we estimate coefficients as
$\hat{\beta}|\lambda = \arg \min_\beta \|y - X\beta \|_2^2 + \|\lambda \beta\|_2^2 \qquad \qquad(1)$
for a given $\lambda$.
If I wanted the ...
0
votes
0answers
152 views
r - Random slopes regression THROUGH THE ORIGIN (0 intercept)
I am in search of a fixed intercept (through the origin) random slopes model. However, instead of additive errors, I want the SLOPE to be the source of random error.
The problem is this: for a ...
1
vote
1answer
73 views
Constraining a model to output only a range between 1 to 100 (without using logistic regression)
Currently, I am modelling the results of game theory experiments where two parties negotiate to divide a fixed sum of money between themselves. Y is recorded as the percentage of the money which the ...
1
vote
0answers
26 views
Constrain multiple Regression for symmetric Stiffness
I want to do a 3-degree polynomial regression like
$$y_i=a_0+\sum_{u=1}^{k} a_{u}x_{i,u}+\sum_{1\leq u \leq v \leq k} a_{u,v} x_{i,u}x_{i,v}+\sum_{1\leq u \leq v\leq w \leq k}a_{u,v,w}x_{i,u}x_{i,v}...
1
vote
2answers
351 views
Constrained (regularized/damped) solution of system of linear equations in Matlab
I have an overdetermined system of linear equations A*X=B. Where X is a column matrix with N unknowns. The least square solution ...
2
votes
0answers
152 views
NNLS convergence proof
I was trying to understand why the nonnegative least squares (NNLS) algorithm of Lawson & Hanson converges to a solution of
$$
\min f_{0}(x) = \| Cx - d \|^{2}, \\
\text{s.t.} \ x \geq 0.
$$
...
4
votes
3answers
880 views
regression with constraints
I have some domain knowledge I want to use in a regression problem.
Problem statement
The dependent variable $y$ is continuous.
The independent variables are $x_1$ and $x_2$.
Variable $x_1$ is ...
1
vote
1answer
2k views
how to use gradient descent to solve ridge regression with a positivity constraint?
I am working on ridge regression at this moment:
$$\sum{(y_{i}-\beta\cdot X_{i})}^{2} + \lambda \cdot\beta^{2}$$
with an additional constraint:
for some $\beta_{m} > 0 $ and some $\beta_{n}<...
1
vote
1answer
28 views
Restricting model parameters in cumulative logit model (or any ordinal model)
Imagine a model where the linear predictor of the cumulative response probabilities are equal to:
$$y = b_0 + b_1x_1 + b_2x_2 + b_3x_1z_1 + b_4x_2z_1$$
Where $x_1$ and $x_2$ represent a 3 level ...
2
votes
1answer
133 views
Restricted OLS problem
Here is my problem:
Let $1\leq l<k$ be integers and write the design matrix $X$ as $X=[X_1\; X_2]$, where $X_1$ is an $n\times l$-dimensional matrix, and where $X_2$ is an $n\times (k-l)$-...
2
votes
1answer
246 views
Performance of linear least squares regression subject to inequality (bounded interval) constraints on parameters
Consider the following model:
$${\bf y} = {\bf X}{\bf b} + {\bf e}$$
where ${\bf y}, {\bf n}\in {\cal R}^m$, ${\bf b}\in{\cal R}^n$, and ${\bf X}\in{\cal R}^{m \times n}$ where $m>n = {\rm rank}(...
3
votes
1answer
399 views
Linear model with constraints on coefficients in terms of ratios
How to fit a linear model (such as lm() in R) of the form:
$$y = a_1 x_1+ a_2 x_2+ a_3 x_3 + a_4 x_4 + a_5 x_5 + \epsilon$$
with the constraint that : $a_2a_5 = ...
2
votes
0answers
37 views
Reversing constrained PCA Regressions while maintaining constraint [closed]
I have an input matrix X that is 1000x10 and a response Y. To reduce the dimensions, I run PCA, select a certain number of factors (say 2) and transform my X to now be 1000x2. I then scale Y and run ...
1
vote
0answers
152 views
How can I constrain the GLM in R to fit the model between measured upper and lower values?
I have a data set for which the explanatory variable is being compared against two response variables, fairly standard. However, my task is to make the fit of the model be constrained between the ...
4
votes
1answer
3k views
How do I fit a constrained regression in R so all coefficients are positive and above 0 [duplicate]
I am trying to understand how to solve the below quadratic program:
This is my model Y=π1X1+π2X2+π3X3+ε,
My constraints are:
- all weights ...
1
vote
0answers
122 views
Constraining regression coefficients using inequalities the easy way?
I want to do something very simple. I have a regression:
y = b0 + b1*X1 + b2*X2 + b3*X3
and the resulting coefficients must be constrained by:
...
3
votes
1answer
154 views
linear regression with partially known coefficients
I wonder if there some existing work of Linear Regression or Logistic Regression with partially known coefficients ($\beta$).
For a linear regression, $Y=X\beta$, when we already have knowledge ...
2
votes
1answer
481 views
1d linear regression with inequality constraint
I want to solve a simple 1D linear regression problem:
$\mathbf{y} = m \mathbf{x} + c$
such that $m_{low} < m < m_{high}$ and $c_{low} < c < c_{high}$.
How can I solve this problem? ...
0
votes
1answer
201 views
Does constrained EM algorithm work with bad initial inputs?
When trying to perform constrained optimization using EM algorithm, does EM work if the initial solution (x0) violates the constraints?
2
votes
1answer
2k views
Interpreting Canonical Correspondence Analysis (CCA) Inertia - in Vegan
I am wanting to know if I can use the ratio of Constrained/Total Inertia in my CCA to describe 'The variability explained by my constraining variables'. I am asking because I've seen different ...
1
vote
2answers
576 views
How to constrain regression coefficient of two variables to have opposite sign?
I am running a simple linear regression with a few variables but the meanings of the variables are such that certain pairs of variables should have coefficients with opposite signs. How should I ...
5
votes
1answer
1k views
Linear regression polynomial slope constraint in R
My problem is how to find the best decreasing 3rd degree polynomial regression in R.
I have data, lets say
...
3
votes
0answers
717 views
Regularized Linear Regression with specific parameter constraints in R
Using R, I can only find tools for performing L1 and/or L2 regularized linear regression (lars, glmnet) and tools for constrained linear regression (quadprog , or lsei {limSolve} , where the ...
