# Questions tagged [convex]

A convex set includes all points lying between any two points from the set. A convex function on such a set is a function lying below any straight line connecting two points from its graph. Convex optimization is concerned with searching for the minimum of such a function.

142 questions
Filter by
Sorted by
Tagged with
13 views

### What is the Best Neural Network architecture to estimate a convex function?

I am currently working on a Q learning algorithm for multi-agent systems and sub-classes of Dec-POMDPs .. It has been shown before that the Q value at any time step can be reduced to a piecewise ...
18 views

### How is the objective function of the different flavors of GARCH different?

How does the objective function/likelihood function of these different GARCH variations differ? Is it convex in all cases? Knowing convexity tells me whether some are not possible to find a globally ...
• 285
1 vote
32 views

### Duality gap calculation in Scikit-learn implementation of Lasso

I am writing a custom variation of Lasso regression, using sklearn's Lasso implementation as a "source of inspiration". And I don't quite understand the very last line in the calculation of ...
• 308
1 vote
28 views

33 views

### from unconstrained to constrained convex optimization

Perhaps a silly question, but I have a Legendre-Fenchel-type optimization $$\psi^{*}(y) = \max \limits_x \, \langle x,y \rangle - \lambda \, \psi(x)$$ for convex $\psi(x)$ and $\lambda > 0$, ...
• 1
1 vote
17 views

### Can we generate HPD regions from MCMC draws using convex hulls?

I thought of a procedure to generate high probability density regions with probability $1-\alpha$ from $n$ MCMC draws: Find the $\lfloor(1-\alpha)\cdot n\rfloor$ draws with the largest probability ...
• 2,536
31 views

### Speeding up an optimization involving matrix products in CVXR

I have an optimization problem where I need to minimize $$-\log \det(U^T \text{diag}(p) U + V^T\text{diag}(1 - p)V)$$ where $p$ is a vector of probabilities, i.e. $0 \leq p_i \leq 1$, and $U$ and $V$ ...
79 views

### Cost function of neural networks can be non-convex, then why do we use it?

I saw a thread here (Cost function of neural network is non-convex?). After I read this, I am really confused. I am wondering that if the cost function is not convex, and we do backpropagation, then ...
• 103
31 views

• 3
13 views

### Clarification needed for a proof step in the paper "Perceptron Mistake Bounds"

I was trying to understand the section 3.1 L1 norm mistake bound (for non-separable case). In the proof of theorem 2, there is a step that takes into account the property of convexity and derives an ...
• 1
52 views

### Gradient descent finds local minima for a problem that can be formulated as a convex problem

I am trying to find $$\min_W \|Y-XW \|_F^2$$ $$s.t. \exists ij, W_{ij}\geq0$$ where X is input data and Y is the output data we try to fit to. This is a convex optimization problem that can be ...
• 261
39 views

### Does the existence of gradient in any function necessarily imply the existence of a subgradient at that point?

First , I apologize if the question is not supposed to be here, or if it is off topic for the subjects dealt with in here. I was reading on subgradients, with respect to convex functions in the ...
• 313
13 views

### Convergence analysis for federated learning using DNN model

In convergence analysis of federated learning (FL), usually, we have an assumption that the loss function is strongly convex. However, when the loss function model is non-convex, e.g., using DNN, I ...
• 101
93 views

### Convergence of CAVI(Coordinate Ascent Variational Inference)

I was reading several resources on variational inference, and most of them stated that the CAVI algorithm converges to local maximum, and Bishop's textbook stated that the convergence is guaranteed as ...
1 vote
54 views

### Can we benefit from a convex loss function when optimizing a neural network?

Many existing loss functions are convex since they are easy to optimize. However, they are only convex with respect to the output $y$, not to parameter $\theta$ of a neural network, or any other non-...
36 views

### Find an extreme point (of a convex polytope) with the minimum of a quadratic cost function

For example, I am trying to do the following: Doubly stochastic matrices (i.e. matrices whose rows and columns sum to 1) form a polytope with permutation matrices as extreme points (Birkhoff–von ...
• 261
44 views

### Log convex function is actually log concave (Pattern Recognition and Machine Learning)

In Pattern Recognition and Machine Learning Ch 6.4.6 at the bottom of page 316 the author states that $p(a_N|t_N)$ is log convex. The author states that: $$-\nabla \nabla \Psi(a_N)=W_N+C_N^{-1}$$ is ...
219 views

### Examples of strongly convex loss functions

This is a reference request. Strong convexity of the loss function is often used in theoretical analyses of convex optimisation for machine learning. My question is, are there important / widely used ...
• 736
31 views

• 3,089
79 views

### Is this objective function convex? [closed]

Given that $F(x)$ is the cumulative distribution function (CDF) of continuous random variable $X$, is $$\frac{\int_0^\infty 1-F(x) dx}{\int_{-\infty}^0 F(x) dx}$$ convex? or is it non-convex/concave? ...
• 3,089