Questions tagged [constrained-optimization]

The tag has no usage guidance.

Filter by
Sorted by
Tagged with
3 votes
1 answer
47 views

Einstein notation $-$ or another $-$ to denote constraints in high dimensional ILP problems

When discussing marginal sums of arrays in 3 dimensions or more, is it customary in the statistical and/or data science communities to use the Einstein summation convention? Is some other form ...
0 votes
0 answers
17 views

Enforce inequality and order constraints in R optim function

I have a function which I am using to calculate the sum of square differences between a 3 component normal mixture distribution function to a set of given quantiles. I want to use this function to ...
0 votes
0 answers
13 views

Confidence interval of parameters under constrained optimization in R

I am using the constrOptim.nl function from the R package alabama to optimize an exponential function with three parameters. Is ...
0 votes
0 answers
11 views

What is the best way to visualize historical product stocks across 10s of shops in corolation with demand?

Imagine we have the following problem, we have 50 different shops, each sell 6 of our items. at the beginning of the month we ship more of those items to the shops based on last month's demand. ...
  • 101
4 votes
1 answer
78 views

Handling multicollinearity with Restricted Least Squares

The dummy variable trap - including a dummy variable for every category and including a constant term in the regression together guarantees perfect multicollinearity - is most commonly resolved by ...
  • 73
0 votes
0 answers
32 views

Minimize cost given working days/hours/required hours per job/start & end dates

I am trying to solve an optimization problem on Excel or Python. My constraints are the available number of employees (=20), the number of working days/month (=20), the number of working hours/day (=8)...
1 vote
0 answers
27 views

CVXPY PSD constraint not working

I am using CVXPY to solve for a PSD matrix, example as follows: ...
2 votes
0 answers
27 views

Train a model subject to max error

I would like to train a neural network by minimizing a loss over samples (as usual), but doing so in a way that the maximum error is bounded. What options do I have? Some that come to mind are: ...
0 votes
0 answers
45 views

How to ensure that the solution to an optimization problem is a probability distribution? [duplicate]

How to ensure that the solution to an optimisation problem is a probability distribution? For example, assume we minimise over a distribution $p$. We must ensure that $\int p(x) \,dx=1 $. But why don'...
  • 27
3 votes
1 answer
41 views

Is standard gradient descent possible when we have constrained parameters?

Is it true that standard gradient descent algorithm (be it batch or mini batch or stochastic) cannot be used when we have certain constraint on parameters? If yes, why is it so? Is it because gradient ...
  • 161
0 votes
1 answer
37 views

deriving the optimal distribution

Let the input variable $X \in \mathcal{X}$ and the target variable $Y \in \mathcal{Y}$. For a fixed hypothesis $h \in \mathcal{H}$ I want to solve \begin{equation} \min_{p(X,Y)} \int_{\mathcal{X}}\...
  • 27
2 votes
0 answers
45 views

LASSO and duality theorem

I am confused with Lagrange duality theorem. Let us consider the problem $$ \hat{\beta} = \underset{\beta \in \mathbb{R}^{n}}{\arg \min} \left[\sum_{i=1}^{n}(y_{i} - \beta_{i})^{2} + \lambda \sum_{i=...
  • 73
1 vote
0 answers
24 views

alternative solution to fussed lasso

The question is related to strange result from fused lasso estimator Let us consider fussed lasso estimator: $$ \hat{\beta}^{FL} = \underset{\beta \in \mathbb{R}^{n}}{\arg \min} [(y_{i} - \beta_{i})^{...
  • 73
4 votes
1 answer
79 views

strange result from fused lasso estimator

Let us consider the following estimator: $$ \hat{\beta}^{F} = \underset{\beta \in \mathbb{R}^{n}}{\arg \min} (y_{i} - \beta_{i})^{2} + \lambda_{1} \sum_{i=1}^{n-1}|\beta_{i} - \beta_{i+1}|, $$ which ...
  • 73
0 votes
0 answers
17 views

Nonlinear constrained optimization for a CIR model

I want to calibrate a CIR model which is commonly used to model the evolution of interest rates. Briefly speaking, we know that its dynamics is of the form \begin{equation} r_t = \kappa (\theta - r_t) ...
0 votes
0 answers
8 views

Encoding knowledge of data lying in the union of hyperplanes using differentiable optimization layers

I know that a way to possibly encode prior knowledge into neural networks training is by using differentiable optimization layers (paper). I'm in the following situation, and I'm wondering if it could ...
3 votes
1 answer
82 views

How does removal of symmetry (e.g. via constraints) in a Bayesian optimization search space affect search efficiency?

There are many examples of search space symmetry in real-world optimization problems in the physical sciences. To motivate this, here are some that come to mind: When optimizing a formulation such as ...
0 votes
0 answers
9 views

Constraining reconstructed vectors to lie in a hyperplane ina VAE

I'm trying to add a linear constraint to my variational autoencoder model. Let's say that my input is made of two concatenated vectors: $\textbf{x} = \textbf{t} \oplus \textbf{y}$ where (for example) ...
2 votes
0 answers
33 views

Covariance matrix of beta coefficients for constrained multiple regression

I have a linear least-squares problem with constraints that two of the coefficients must be non-negative. For a typical (unconstrained) least squares estimation, I know that the variance-covariance ...
  • 21
0 votes
0 answers
38 views

Projected Gradient Descent for neural networks

One of the most widely used optimization methods for neural networks is (stochastic) gradient descent. When encountering constrained problems, a standard modification of gradient descent consists of ...
  • 105
1 vote
0 answers
322 views

`Error: L-BFGS-B needs finite values of 'fn'` of Complex Objective Function

I'm trying to run R's maximum likelihood estimation function (stats4::mle), over a likelihood function in Free Shipping Is Not ...
  • 21
0 votes
0 answers
50 views

Why SCM weights *should* sum to 1?

When we use synthetic controls, we consider $j=2, \dots, J+1$ units across $t \in \{T_{-} \ldots T_0 \ldots T_{+}\} \in \mathbb{Z}$ pre/post event-time periods. Abadie et. al. make the point that to ...
2 votes
1 answer
40 views

KKT Conditions for thresholds?

My main question is that when I use Lagrange Multipliers/KKT conditions to perform optimization with threshold constraints, I seem to get contradictory FOC. Here is a characteristic example: take an ...
  • 23
1 vote
0 answers
346 views

How do you use pytorch to solve strictly constrained optimization problems? [closed]

I am trying to solve the following problem using pytorch: given a six sided die whose average roll is known to be 4.5, what is the maximum entropy distribution for the faces? (Note: I know a bunch of ...
0 votes
0 answers
46 views

Speeding up an optimization involving matrix products in CVXR

I have an optimization problem where I need to minimize $$-\log \det(U^T \text{diag}(p) U + V^T\text{diag}(1 - p)V)$$ where $p$ is a vector of probabilities, i.e. $0 \leq p_i \leq 1$, and $U$ and $V$ ...
's user avatar
0 votes
0 answers
76 views

Is sign constrained LASSO regression a valid approach when dealing with a small dataset?

Context: In a dataset (n=250/events=60) we are trying to predict a time to event outcome using lasso cox regression as specified in glmnet. In evaluating our model, 2/5 predictors show a ...
1 vote
0 answers
76 views

Discrete Bayes Net learning under parameter constraints

What is some relevant research available on estimating the parameters of a Bayes Net (with known structure) when there are known constraints on conditional and marginal probabilities? For example, ...
  • 1,148
1 vote
1 answer
87 views

How do linear constraints affect the convexity of my OLS-like optimisation problem?

I would like to augment a linear regression (so a convex OLS problem) with some additional constraints on the coefficients to match the subject I'm working on. Having $x\in \mathbb{R}^n$, the solution ...
  • 11
0 votes
0 answers
67 views

Applying constraints within a neural network?

I need to solve a multi-objective problem. I would understand if there is any kind of possibility to cope this issue through a neural network. Hypothetically, I need to put some constraints within ...
2 votes
1 answer
18 views

Optimal Feature Engeneering creation: best optimization method?

basically I would like to solve this problem: (1) say I have N features that I want to transform with a generic f(x, theta) ...
  • 219
2 votes
0 answers
38 views

Can I use xboost as objective function in an optimization problem?

I am working on a marketing optimization problem, where the goal is maximize profit by optimally allocating spend to different products. Constraint is getting at least 1 Million revenue. As a first ...
  • 747
1 vote
0 answers
30 views

Solving coefficient sum constrained elastic net with quadratic objective term

I am looking for an algorithm to solve an equality constrained elastic net. There are two adaptations I need to make to the standard elastic net. First the objective function includes a quadratic ...
  • 11
2 votes
0 answers
56 views

Gradient descent finds local minima for a problem that can be formulated as a convex problem

I am trying to find $$ \min_W \|Y-XW \|_F^2$$ $$s.t. \exists ij, W_{ij}\geq0 $$ where X is input data and Y is the output data we try to fit to. This is a convex optimization problem that can be ...
  • 271
1 vote
0 answers
43 views

What is the easiest way to solve the following constrained optimization problem in python? [closed]

So, I have a simple optimization problem, I have a cloud of 3D points, $x = (x_1, x_2\dots x_n) $ with $x_i = [x_{i1}, x_{i2}, x_{i3}]$. And I want to minimize the mean squared error between the ...
  • 147
1 vote
1 answer
226 views

Constrained optimization with gradient descent

Suppose I want to maximize the likelihood $L(\theta_1, \theta_2)$ for some constraint for example $\theta_1 + \theta_2 = 1$ and no other constraints Can I just replace $\theta_2$ by $1 - \theta_1$ in ...
  • 65
1 vote
0 answers
47 views

Fitting sums of Gaussian-like density functions

I acknowledge that similar question has been asked a couple years ago, yet it still seems unresolved. Ignoring the domain knowledge, the statistical problem behind is to fit a non-linear regression ...
0 votes
2 answers
152 views

How to adapt a linear time Newton-Raphson numerical method for an optimisation problem with positivity constraints?

I would appreciate some assistance in understanding how I can adapt a linear time Newton-Raphson root finding algorithm for unconstrained optimisation, to solve a problem where I introduce positivity ...
  • 2,236
1 vote
1 answer
23 views

How to formally define the set of function that an guarantee a unique optima in constraint maximisation problem

We know that a strict concave function can guarantee a unique maximum. But in a constraint maximisation problem, there can still exists a unique maximum when the function is not concave. My question ...
  • 359
-1 votes
1 answer
19 views

Match a Sequence to a bigger Sequence

I am trying to solve the following problem: Say I have a small sequence $\sigma = (σ_{1}, σ_{2}, ..., σ_{i}, ..., σ_{N})$ and a larger $\hat{\sigma} = (\hat{σ}_{1}, \hat{σ}_{2}, ..., \hat{σ}_{j}, ..., ...
0 votes
2 answers
347 views

How to do gradient descent when parameter is positive definite matrix

So, suppose I have an objective function $\mathcal{L}(\Sigma)$ where $\Sigma$ is a positive definite matrix. Now, I want to optimize this function using gradient descent. Now, I think if I compute the ...
4 votes
2 answers
180 views

In optimization, is there a distinction between "implicit/natural" and "explicit/designed" constraints?

For example, I wish to optimization a function which has a log term $\log(x)$ Now the very presence of the log term induces a constraint which says $x > 0$. The case $x = 0 $ might be a bit ...
  • 307
15 votes
2 answers
3k views

MLE for normal distribution with restrictive parameters

Suppose that $X_1, . . . , X_n$, $n\geq 2$, is a sample from a $N(\mu,\sigma^2)$ distribution. Suppose $\mu$ and $\sigma^2$ are both known to be nonnegative but otherwise unspecified. Now, I want to ...
  • 561
1 vote
1 answer
332 views

Fitting Gaussian mixture model with constraints (eg. mu1<mu2) in Python

My question is similar to this one, but while the OP there has constrains such as mu1 being <=0 and mu2 being >=0, my constraints are following: It's a three component mixture model. mu1 < ...
4 votes
1 answer
613 views

Is it possible to optimize correlation coefficient under linear constraint?

I am new to optimization and recently bump into a problem where I have to optimize the correlation coefficient of a series of values with the absolute value of another vector under the linear ...
4 votes
2 answers
115 views

MSE of correlations

These might be dumb questions but I am having trouble to wrap my head around of a particular problem. I have a sparse count matrix $G $ that I want to optimize which is $N \times p$. Also, I have ...
  • 93
2 votes
1 answer
34 views

Resulting shapes when partitioning the constraint matrix $\boldsymbol{A}$ in linear programming

\begin{equation} \boldsymbol{A} = \begin{bmatrix} {1}_n^\top \otimes \mathbb{I}_m \\ \mathbb{I}_n \otimes {1}_m^\top \end{bmatrix} \in \mathbb{R}^{(m+n)\times mn} \end{equation} If the above matrix ...
  • 3,199
1 vote
1 answer
48 views

L1 norm for compressed sensing

In a paper, I read that this is one problem with the L1 norm for Compressed sensing (the L1 norm function is not smooth). I am curious that why is that a problem? Can anyone explain why is it so ...
  • 11
0 votes
0 answers
36 views

Combining two terms into a quadratic form

I have an objective function defined by $ min_{Y_{t}} \hspace{2mm} ||X_{t} - Y_{t}D_{t}^{T}||_{F}^{2} + \lambda_{2}\sum_{i,j} w_{i,j}||\mathbf{y}_{i} - \mathbf{p}_{j}^{t}||_{2}^{2}$ where capital $T$ (...
  • 1,596
0 votes
0 answers
33 views

ML algorithm to create an optimal photo portfolio

I would like to train an ML algorithm that would help create a best subset of 3 photos from a set of N photos. I'm wondering what kind of model could be used for this kind of portfolio/deck ...
  • 143
2 votes
1 answer
47 views

can we perform sub-gradient updates in mini-batches

We are already aware that in case the data is quite bulky, mini-batch gradient descent based approaches may be applied. These approaches load a mini-batch of data, compute the loss on this batch, and ...
  • 1,596