# Questions tagged [constrained-optimization]

The tag has no usage guidance.

73 questions
Filter by
Sorted by
Tagged with
20 views

### Constrained Cholesky Decomposition

Suppose that I have an $(n\times 1)$ vector of random variables, $\varepsilon$. However, I know that $k$ linear combinations of $\varepsilon$ are 0. Specifically, I know that for a $(k\times n)$ ...
42 views

• 308
24 views

### Implementations for non-negative regression in generalized linear models [duplicate]

What are some package for generalized linear models that give non-negative regression coefficients without regularization? I checked the thread (Nonnegative generalized linear model), but many ...
• 21
1 vote
59 views

### Relaxed non-negative least squares

I am reconstructing a probability vector from data using non-negative least squares: $$\sum_\alpha \left(\pi_\alpha - \sum_i W_{\alpha i}p_i\right)^2\rightarrow \min,\\ p_i\geq 0,\sum_i p_i=1$$ ...
• 4,439
64 views

### Einstein notation $-$ or another $-$ to denote constraints in high dimensional ILP problems

When discussing marginal sums of arrays in 3 dimensions or more, is it customary in the statistical and/or data science communities to use the Einstein summation convention? Is some other form ...
• 2,244
357 views

### Handling multicollinearity with Restricted Least Squares

The dummy variable trap - including a dummy variable for every category and including a constant term in the regression together guarantees perfect multicollinearity - is most commonly resolved by ...
• 73
233 views

### CVXPY PSD constraint not working

I am using CVXPY to solve for a PSD matrix, example as follows: ...
214 views

### Train a model subject to max error

I would like to train a neural network by minimizing a loss over samples (as usual), but doing so in a way that the maximum error is bounded. What options do I have? Some that come to mind are: ...
45 views

### How to ensure that the solution to an optimization problem is a probability distribution? [duplicate]

How to ensure that the solution to an optimisation problem is a probability distribution? For example, assume we minimise over a distribution $p$. We must ensure that $\int p(x) \,dx=1$. But why don'...
• 127
201 views

### Is standard gradient descent possible when we have constrained parameters?

Is it true that standard gradient descent algorithm (be it batch or mini batch or stochastic) cannot be used when we have certain constraint on parameters? If yes, why is it so? Is it because gradient ...
• 421
49 views

### deriving the optimal distribution

Let the input variable $X \in \mathcal{X}$ and the target variable $Y \in \mathcal{Y}$. For a fixed hypothesis $h \in \mathcal{H}$ I want to solve \min_{p(X,Y)} \int_{\mathcal{X}}\...
• 127
115 views

### strange result from fused lasso estimator

Let us consider the following estimator: $$\hat{\beta}^{F} = \underset{\beta \in \mathbb{R}^{n}}{\arg \min} (y_{i} - \beta_{i})^{2} + \lambda_{1} \sum_{i=1}^{n-1}|\beta_{i} - \beta_{i+1}|,$$ which ...
• 43
165 views

### How does removal of symmetry (e.g. via constraints) in a Bayesian optimization search space affect search efficiency?

There are many examples of search space symmetry in real-world optimization problems in the physical sciences. To motivate this, here are some that come to mind: When optimizing a formulation such as ...
• 69
77 views

### Covariance matrix of beta coefficients for constrained multiple regression

I have a linear least-squares problem with constraints that two of the coefficients must be non-negative. For a typical (unconstrained) least squares estimation, I know that the variance-covariance ...
• 21
1 vote
1k views

### Error: L-BFGS-B needs finite values of 'fn' of Complex Objective Function

I'm trying to run R's maximum likelihood estimation function (stats4::mle), over a likelihood function in Free Shipping Is Not ...
• 21
149 views

### Why SCM weights *should* sum to 1?

When we use synthetic controls, we consider $j=2, \dots, J+1$ units across $t \in \{T_{-} \ldots T_0 \ldots T_{+}\} \in \mathbb{Z}$ pre/post event-time periods. Abadie et. al. make the point that to ...
97 views

### KKT Conditions for thresholds?

My main question is that when I use Lagrange Multipliers/KKT conditions to perform optimization with threshold constraints, I seem to get contradictory FOC. Here is a characteristic example: take an ...
• 115
1 vote
865 views

### How do you use pytorch to solve strictly constrained optimization problems? [closed]

I am trying to solve the following problem using pytorch: given a six sided die whose average roll is known to be 4.5, what is the maximum entropy distribution for the faces? (Note: I know a bunch of ...
• 221
136 views

### Speeding up an optimization involving matrix products in CVXR

I have an optimization problem where I need to minimize $$-\log \det(U^T \text{diag}(p) U + V^T\text{diag}(1 - p)V)$$ where $p$ is a vector of probabilities, i.e. $0 \leq p_i \leq 1$, and $U$ and $V$ ...
1 vote
84 views

### Discrete Bayes Net learning under parameter constraints

What is some relevant research available on estimating the parameters of a Bayes Net (with known structure) when there are known constraints on conditional and marginal probabilities? For example, ...
• 1,158
1 vote
187 views

### How do linear constraints affect the convexity of my OLS-like optimisation problem?

I would like to augment a linear regression (so a convex OLS problem) with some additional constraints on the coefficients to match the subject I'm working on. Having $x\in \mathbb{R}^n$, the solution ...
• 11