Questions tagged [constrained-regression]
constrained regression is (linear or nonlinear) regression models with further constraints on the coefficients. That could be non-negativity constraints, constraints on the norm of the coefficient vector or otherwise.
114
questions
0
votes
1answer
27 views
Minimizing $L_2$ norm with constrained residual sum of squares (RSS)
I have some complex-valued time-series data, $y \in \mathbb{C}^n$ - a signal with additive Gaussian white noise. The goal is to find the Fourier coefficients of this signal.
Ideally, you would just do ...
0
votes
0answers
44 views
How to get standard errors of linear regression coefficients subject to linear equality constraints
I have a regression problem with multiple sets of sum-to-one categorical features (e.g. X1=Male, X2=Female, X3=Weekday, X4=Weekend), intercept needs to be included as well as some restrictions on the ...
0
votes
0answers
15 views
Neural network regression with a constrained dependent variable
I'm using a multi-layer perceptron model to predict the cumulative number of boardings and alightings from a bus travelling on a standard route based on previous cumulative boarding and alighting ...
1
vote
0answers
20 views
Difference between glmnet and nnls for non-negative least squares in R
I'm trying to do some non-negative linear regressions in R, and I found in the blog here https://www.r-bloggers.com/2019/11/non-negative-least-squares/ that either the package ...
3
votes
2answers
126 views
How to fit a constrained regression in R?
My regression:
$Y = α + β_1X_1 + β_2X_2 + β_3X_3 + β_4X_4 + β_5X_5 + β_6X_6 + β_7X_7 + β_8X_8 + β_9X_9 + β_{10}X_{10} + β_{11}X_{11} + β_{12}X_{12} + β_{13}X_{13} + β_{14}X_{14} + ε$
Constraints that ...
0
votes
0answers
12 views
Constrained least squares for standardization of dataset
I have a set of correlation matrices $\mathbf{\Sigma}_{i,j}$ where $i$ is the $i^{th}$ dataset and $j$ is the $j^{th}$ sample of the $i^{th}$ dataset. I am trying to standardize correlation matrix ...
0
votes
1answer
106 views
Can it be proven that non-negative least squares is a perfectly convex problem?
Can it be proven that NNLS is a perfectly convex problem?
Myre writes in section 3 of his TNT-NN manuscript:
The NNLS objective function is a convex quadratic function with
linear inequalities as ...
1
vote
1answer
33 views
Numerical solution to the constrained ridge regression
The constrained ridge regression problem is of the form:
$\arg\min_{\|\beta\|_2\le t}\|X\beta-y\|_2$. Given a matrix $X$, a vector $y$ and the constrain parameter $t$, how do you solve it numerically?
...
0
votes
0answers
12 views
How to fit a constrained harmonic fit in R to mean daily temperature data?
I am trying to reproduce the statistics procedures for creating daily normal temperatures according to this paper.
I was provided code in IDL (a Fortran based scientific language), but it is so ...
0
votes
0answers
25 views
Constraining regression coefficient to non-negative [duplicate]
I have a regression problem where I don't want the coefficients to be negative. Is setting negative coefficients of OLS to zero the same as constraining the coefficient to be non-zero and solving it ...
2
votes
0answers
42 views
How to get accurate estimates on Neural Networks Hessian?
I need to get not only accurate estimates on the neural network output itself but also on its second order derivatives in order to use the NN for optimization problems. With Adam optimizer I can't get ...
1
vote
0answers
92 views
Linear Regression with constraint(s): need s.e., t-stat, and p-values of the regressors
I am performing a linear regression and what I need is (1) to constrain the sum of the regression coefficients to 1, and (2) to constrain the sum of regression coefficients to 1 AND each regression ...
0
votes
0answers
43 views
Query on constrained regressions using -mgcv- and -pcls- in R
Bert Gunter from the R-help list directed me to ask this query here. Basically, I'm puzzled!
How is it that this constrained regression routine using -pcls- runs satisfactorily (courtesy of Tian Zheng)...
0
votes
0answers
38 views
Perform regression on a set of equations with additional constraints
I want to analyse the composition of a ternary mixture (a mixture containing of three components) by combining measurements of the mixture's density (denoted $\rho$) and its refractive index (denoted $...
0
votes
0answers
18 views
Can regression coefficients be guaranteed to be normalized?
When solving a multivariate linear regression problem of the form $A\vec{x}=\vec{y}$ where $A$ and $\vec{y}$ are known. Is there any from of preprocessing or scaling that can be done $A$ and or $\vec{...
2
votes
3answers
809 views
How to test if sum of two coefficients of ols model is greater than zero using R?
The regression model is:
y = b0 +b1x1 + b2x2 + b3x3 + e
I want to test if b1 + b2 > 0.
the R package ...
2
votes
1answer
43 views
How can this model be fit using R's lm function?
I have data that consists of a grouping variable $\tt{grp}$, a predictor $\tt{x}$, and a response $\tt{y}$. There are three groups. I want to fit to this data the model $y = \alpha_i + (\beta + \...
0
votes
1answer
47 views
Constrained optimization of joint Bernoulli density parametrized by a neural network
I want to learn a joint distribution on $n$ Bernoulli random variables conditioned on a random variable $b\sim D$ and parametrized by a neural network, $f$:
$$p(A = (a_1, \ldots, a_n)|b) = f_{\Lambda}(...
3
votes
0answers
27 views
Regression with coefficients having a multiplicative relationship
Assume I need to calibrate a linear model of the form shown here:
$$Y\sim\beta_0+\beta_1X_1+\beta_2X_2$$
where the $\beta$ terms are the coefficients and the $X$ terms are the independent variables. ...
1
vote
0answers
29 views
Least-squares fit of explicit parabolic sheet to scattered data points [duplicate]
For a given set of data points
$$\{(x_i, y_i, z_i)\}$$
there exists some
$$f_{ABC}(x,y)=Ax^2+Bxy+Cy^2$$
that minimizes
$$\sum_i(f_{ABC}(x_i,y_i)-z_i)^2$$
$A$, $B$, and $C$ can be found quickly ...
5
votes
2answers
795 views
Monotonic splines in Python [closed]
I am trying to find a procedure to fit data monotonically in Python.
The data won’t be necessarily monotonic. I just would like to achieve a monotonic fit because of theoretical assumptions.
I ...
4
votes
1answer
201 views
How do you apply constrains on parameters in Bayesian modeling?
In Frequentist modeling, I know how to fit and interpret models with constraints on the parameter space.
Let's say I'm fitting a $N(\mu, \sigma^2)$ distribution to my data $x_1, \dots, x_n$, and I ...
4
votes
0answers
60 views
Linear regression of higher order polynomial with slope constraint
I am trying to constrain the coefficients on a higher order polynomial (let's say an order 6) for the curve to be decreasing.
I have found this link, where the fitting of a 3rd order polynomial is ...
3
votes
1answer
61 views
How to fit several linear models with linear intercept-slope constrain?
I have a data where several groups exist: $i = 1, 2, 3...n$ ($n$ about 600 groups in total).
For group $i$ theory predicts a linear relation $y = a_i + b_i * x$ with $a_i$ and $b_i$ intercept and ...
2
votes
2answers
290 views
Adding limits to regression coefficients
For my problem, I have data that contains daily observations of the total time and the volumes of task A completed, task B completed, C, D.. and I am looking to estimate the time it takes to solve ...
0
votes
0answers
103 views
Is there a conjugate prior for normal location family N(x|u,1) such that the mean is always positive?
The conjugate prior to normal location family is usually a normal distribution. However, I want to constrain the mean to be positive.
Is there a conjugate prior to the normal location family $x\sim N(...
3
votes
1answer
66 views
Model/link function to deal with dependent variable in range [-1,1]?
My dependent variable, $Y$, contains values anywhere from -1 to 1 (i.e. it is bounded continuously on the range $[-1,1]$). I know that a regular OLS regression on such a variable would sometimes ...
1
vote
0answers
79 views
Sum of residuals in constrained linear regression
Does sum of residuals == 0 hold even if we add ordinality constraints for the coefficients in a regression model?
It would be great if someone can point out to any literature (paper/presentation/...
0
votes
0answers
113 views
Closed-form solutions for constrained multiple linear regression
Normally a multiple linear regression is unconstrained
$$y=X\beta+\epsilon$$
so that closed-form solutions in the case of data orthogonality ($X^\top X=I$) are
$$\beta=(X^\top X)^{-1} X^\top y$$
...
0
votes
0answers
227 views
Restricted Weighted Linear Regression in R
I have to follwing issue. I would like to run a linear regression imposing a constraint on the weighted coefficients. Let me construct an example:
Consider the following cross-sectional regression
$...
2
votes
2answers
766 views
Predict a vector of values with constraints? [duplicate]
I am aware of a variety of methods for simultaneously predicting multiple outcomes known sometimes as multivariate regression/analysis. However, my situation is a little more special. I am trying to ...
0
votes
1answer
96 views
How can you constrain values to be positive when fitting a model?
I'm currently fitting a model using maximum likelihood estimation on biological data (electroencephalography). Basically, I'm fitting normal distributions to several subsets of data (experimental ...
0
votes
1answer
199 views
least square with equality constraint and singular matrix
I have the following problem to solve:
Ax = b where A is singular.
To resolve this I introduce a condition Cx = d
Even after this, I am not able to solve it using scipy.optimize.minimize as it ...
2
votes
1answer
2k views
Is it possible to apply a monotonicity constraint on a Gaussian process regression fit?
Below is a code using scikit-learn where I simply apply Gaussian process regression (GPR) on a set of observed data to produce an expected fit. I know physically that this curve should be ...
1
vote
0answers
577 views
R - Linear Regression with parameters constraint - contr.sum contrast
I am facing a problem with running a simple OLS regression with two categorical independent variable.
I would like to impose that the sum of the coefficients referred to one variable (x in the ...
3
votes
0answers
92 views
How to run linear regression with constraints in R? [duplicate]
If I have the following data
n<-1000
x1<-rnorm(n,1,1)
x2<-rnorm(n,2,2)
x3<-rnorm(n,3,3)
e<-rnorm(n)
y<-3+0.5*x1+0.2*x2+0.3*x3+e
I want to fit a ...
2
votes
1answer
963 views
method for predicting a curve
I have data on several curves. the data is of the form:
curve_id x y
and there are many x/y pairs for each curve and x is limited to some known range. overall, the curves look quite similar in ...
0
votes
1answer
462 views
polynomial regression in R: how to add hard constraints (go through specified points)? [duplicate]
I'd like to perform polynomial regression on my data (which lies in [0,10]), but I need to ensure that the endpoints of the range are fixed, i.e. that the curve goes through (0,0) and (10,10). So the ...
1
vote
0answers
213 views
nlme ignoring certain control arguments to nlminb
Issue: Do constrained optimization of parameters in nlme::nlme
I'm trying to fit a non-linear mixed effects model using nlme::nlme, which can use 2 optimization schemes: stats::nlminb or stats::nlm.
...
0
votes
1answer
75 views
Confidence limits for constrained penalized log likelihood model
I am estimating parameter $\beta$ as:
\begin{align}
\hat \beta &= \mathop{\mathrm{arg\,max}}_\beta \;\; l(\beta;X,y) - \frac{\lambda}{2}\left(\tilde y-g(\beta,\tilde X)\right)^\prime C^\prime C\...
3
votes
1answer
437 views
Is constrained (nonnegative) least squares a form a regularisation?
Is nonnegative least squares already a form of regularisation? By adding a constraint that $\beta \geq 0$ (the coefficients), does it make sense to add another regularisation term as in LASSO or ridge ...
10
votes
3answers
2k views
Calculating the p-values in a constrained (non-negative) least squares
I have been using Matlab to perform unconstrained least squares (ordinary least squares) and it automatically outputs the coefficients, test statistic and the p-values.
My question is, upon ...
1
vote
1answer
1k views
Constrained Linear Regression with coefficients related by inequality
I have a (slightly simplified) model of the following form:
$Y=c_1X_1 + c_2X_2 + \epsilon$ subject to the constraint $0\leq c_1\leq c_2$.
The distribution of $\epsilon$ is actually not important to ...
0
votes
1answer
653 views
RDA and CCA output- unconstrained inertia 0.00 rank 0
I get an ouput for RDA and for CCA that says that my unconstrained inertia is 0, rank 0. I thought that would be a good thing, meaning that all the variance in the data is explained by my (constrained)...
1
vote
0answers
642 views
Comparing log-transformed and non-log transformed models
Suppose I have the following two models fitted with constrained linear least squares:
Model 1: $Y = \beta_1X_1 + \ldots + \beta_kX_k + \varepsilon$
Model 2: $\log_{10}(Y) = \beta_1\log_{10}(X_1) + \...
3
votes
1answer
200 views
How to perform Least Squares with constraints on a subset of the model coefficients?
For solving an unconstrained LS regression
$$\hat{y}=w_1.x_1+w_2.x_2+w_3.x_3+w_4.x_4 + \epsilon$$
I use the following normal equation:
$$W^*=(X^{\top}X)^{-1}X^{\top}Y $$
If I want to introduce a ...
23
votes
2answers
2k views
The limit of "unit-variance" ridge regression estimator when $\lambda\to\infty$
Consider ridge regression with an additional constraint requiring that $\hat{\mathbf y}$ has unit sum of squares (equivalently, unit variance); if needed, one can assume that $\mathbf y$ has unit sum ...
1
vote
0answers
75 views
How to create a constrained spline meta model in R?
Background
I have an outcome measure I'm trying to predict that is on a scale 0-1000. I've trained (say) M different models (model definition doesn't matter) on overlapping sub-ranges of the data (e....
5
votes
1answer
1k views
What are drawbacks of isotonic regression?
I have been reading about isotonic regression and it seems like a great method that will give one a monotone regression function estimator and, moreover, is free of any tuning parameters.
Why are ...
3
votes
0answers
204 views
Fit big GAM with constraints on linear terms
I'm trying to fit MGCV GAM with constraints on linear terms. I feed a dataset with factor variable G. Number of rows is about 30-40k. Number of model parameters - ...
