# Questions tagged [constrained-optimization]

The tag has no usage guidance.

52 questions
Filter by
Sorted by
Tagged with
47 views

### Einstein notation $-$ or another $-$ to denote constraints in high dimensional ILP problems

When discussing marginal sums of arrays in 3 dimensions or more, is it customary in the statistical and/or data science communities to use the Einstein summation convention? Is some other form ...
• 2,124
17 views

### Enforce inequality and order constraints in R optim function

I have a function which I am using to calculate the sum of square differences between a 3 component normal mixture distribution function to a set of given quantiles. I want to use this function to ...
13 views

### Confidence interval of parameters under constrained optimization in R

I am using the constrOptim.nl function from the R package alabama to optimize an exponential function with three parameters. Is ...
11 views

### What is the best way to visualize historical product stocks across 10s of shops in corolation with demand?

Imagine we have the following problem, we have 50 different shops, each sell 6 of our items. at the beginning of the month we ship more of those items to the shops based on last month's demand. ...
• 101
78 views

### Handling multicollinearity with Restricted Least Squares

The dummy variable trap - including a dummy variable for every category and including a constant term in the regression together guarantees perfect multicollinearity - is most commonly resolved by ...
• 73
32 views

### Minimize cost given working days/hours/required hours per job/start & end dates

I am trying to solve an optimization problem on Excel or Python. My constraints are the available number of employees (=20), the number of working days/month (=20), the number of working hours/day (=8)...
1 vote
27 views

### CVXPY PSD constraint not working

I am using CVXPY to solve for a PSD matrix, example as follows: ...
27 views

### Train a model subject to max error

I would like to train a neural network by minimizing a loss over samples (as usual), but doing so in a way that the maximum error is bounded. What options do I have? Some that come to mind are: ...
45 views

### How to ensure that the solution to an optimization problem is a probability distribution? [duplicate]

How to ensure that the solution to an optimisation problem is a probability distribution? For example, assume we minimise over a distribution $p$. We must ensure that $\int p(x) \,dx=1$. But why don'...
• 27
41 views

### Is standard gradient descent possible when we have constrained parameters?

Is it true that standard gradient descent algorithm (be it batch or mini batch or stochastic) cannot be used when we have certain constraint on parameters? If yes, why is it so? Is it because gradient ...
• 161
37 views

### deriving the optimal distribution

Let the input variable $X \in \mathcal{X}$ and the target variable $Y \in \mathcal{Y}$. For a fixed hypothesis $h \in \mathcal{H}$ I want to solve \begin{equation} \min_{p(X,Y)} \int_{\mathcal{X}}\...
• 27
45 views

• 73
79 views

### strange result from fused lasso estimator

Let us consider the following estimator: $$\hat{\beta}^{F} = \underset{\beta \in \mathbb{R}^{n}}{\arg \min} (y_{i} - \beta_{i})^{2} + \lambda_{1} \sum_{i=1}^{n-1}|\beta_{i} - \beta_{i+1}|,$$ which ...
• 73
17 views

### Nonlinear constrained optimization for a CIR model

I want to calibrate a CIR model which is commonly used to model the evolution of interest rates. Briefly speaking, we know that its dynamics is of the form \begin{equation} r_t = \kappa (\theta - r_t) ...
• 131
8 views

### Encoding knowledge of data lying in the union of hyperplanes using differentiable optimization layers

I know that a way to possibly encode prior knowledge into neural networks training is by using differentiable optimization layers (paper). I'm in the following situation, and I'm wondering if it could ...
• 517
82 views

### How does removal of symmetry (e.g. via constraints) in a Bayesian optimization search space affect search efficiency?

There are many examples of search space symmetry in real-world optimization problems in the physical sciences. To motivate this, here are some that come to mind: When optimizing a formulation such as ...
• 69
9 views

### Constraining reconstructed vectors to lie in a hyperplane ina VAE

I'm trying to add a linear constraint to my variational autoencoder model. Let's say that my input is made of two concatenated vectors: $\textbf{x} = \textbf{t} \oplus \textbf{y}$ where (for example) ...
• 517
33 views

### Covariance matrix of beta coefficients for constrained multiple regression

I have a linear least-squares problem with constraints that two of the coefficients must be non-negative. For a typical (unconstrained) least squares estimation, I know that the variance-covariance ...
• 21
38 views

### Projected Gradient Descent for neural networks

One of the most widely used optimization methods for neural networks is (stochastic) gradient descent. When encountering constrained problems, a standard modification of gradient descent consists of ...
• 105
1 vote
322 views

### Error: L-BFGS-B needs finite values of 'fn' of Complex Objective Function

I'm trying to run R's maximum likelihood estimation function (stats4::mle), over a likelihood function in Free Shipping Is Not ...
• 21
50 views

### Why SCM weights *should* sum to 1?

When we use synthetic controls, we consider $j=2, \dots, J+1$ units across $t \in \{T_{-} \ldots T_0 \ldots T_{+}\} \in \mathbb{Z}$ pre/post event-time periods. Abadie et. al. make the point that to ...
40 views

### KKT Conditions for thresholds?

My main question is that when I use Lagrange Multipliers/KKT conditions to perform optimization with threshold constraints, I seem to get contradictory FOC. Here is a characteristic example: take an ...
• 23
1 vote
346 views

### How do you use pytorch to solve strictly constrained optimization problems? [closed]

I am trying to solve the following problem using pytorch: given a six sided die whose average roll is known to be 4.5, what is the maximum entropy distribution for the faces? (Note: I know a bunch of ...
• 221
46 views

### Speeding up an optimization involving matrix products in CVXR

I have an optimization problem where I need to minimize $$-\log \det(U^T \text{diag}(p) U + V^T\text{diag}(1 - p)V)$$ where $p$ is a vector of probabilities, i.e. $0 \leq p_i \leq 1$, and $U$ and $V$ ... 76 views

### Is sign constrained LASSO regression a valid approach when dealing with a small dataset?

Context: In a dataset (n=250/events=60) we are trying to predict a time to event outcome using lasso cox regression as specified in glmnet. In evaluating our model, 2/5 predictors show a ...
1 vote
76 views

### Discrete Bayes Net learning under parameter constraints

What is some relevant research available on estimating the parameters of a Bayes Net (with known structure) when there are known constraints on conditional and marginal probabilities? For example, ...
• 1,148
1 vote
87 views

### How do linear constraints affect the convexity of my OLS-like optimisation problem?

I would like to augment a linear regression (so a convex OLS problem) with some additional constraints on the coefficients to match the subject I'm working on. Having $x\in \mathbb{R}^n$, the solution ...
• 11
67 views

### Applying constraints within a neural network?

I need to solve a multi-objective problem. I would understand if there is any kind of possibility to cope this issue through a neural network. Hypothetically, I need to put some constraints within ...
18 views

### Optimal Feature Engeneering creation: best optimization method?

basically I would like to solve this problem: (1) say I have N features that I want to transform with a generic f(x, theta) ...
• 219
38 views

### Can I use xboost as objective function in an optimization problem?

I am working on a marketing optimization problem, where the goal is maximize profit by optimally allocating spend to different products. Constraint is getting at least 1 Million revenue. As a first ...
• 747
1 vote
30 views

### Solving coefficient sum constrained elastic net with quadratic objective term

I am looking for an algorithm to solve an equality constrained elastic net. There are two adaptations I need to make to the standard elastic net. First the objective function includes a quadratic ...
• 11
56 views

### Gradient descent finds local minima for a problem that can be formulated as a convex problem

I am trying to find $$\min_W \|Y-XW \|_F^2$$ $$s.t. \exists ij, W_{ij}\geq0$$ where X is input data and Y is the output data we try to fit to. This is a convex optimization problem that can be ...
• 271
1 vote
43 views

### What is the easiest way to solve the following constrained optimization problem in python? [closed]

So, I have a simple optimization problem, I have a cloud of 3D points, $x = (x_1, x_2\dots x_n)$ with $x_i = [x_{i1}, x_{i2}, x_{i3}]$. And I want to minimize the mean squared error between the ...
• 147
1 vote
226 views

### Constrained optimization with gradient descent

Suppose I want to maximize the likelihood $L(\theta_1, \theta_2)$ for some constraint for example $\theta_1 + \theta_2 = 1$ and no other constraints Can I just replace $\theta_2$ by $1 - \theta_1$ in ...
• 65
1 vote
47 views

### Fitting sums of Gaussian-like density functions

I acknowledge that similar question has been asked a couple years ago, yet it still seems unresolved. Ignoring the domain knowledge, the statistical problem behind is to fit a non-linear regression ...
152 views

### How to adapt a linear time Newton-Raphson numerical method for an optimisation problem with positivity constraints?

I would appreciate some assistance in understanding how I can adapt a linear time Newton-Raphson root finding algorithm for unconstrained optimisation, to solve a problem where I introduce positivity ...
• 2,236
1 vote
23 views

### How to formally define the set of function that an guarantee a unique optima in constraint maximisation problem

We know that a strict concave function can guarantee a unique maximum. But in a constraint maximisation problem, there can still exists a unique maximum when the function is not concave. My question ...
• 359
19 views

• 1,596
33 views

### ML algorithm to create an optimal photo portfolio

I would like to train an ML algorithm that would help create a best subset of 3 photos from a set of N photos. I'm wondering what kind of model could be used for this kind of portfolio/deck ...
• 143