# Questions tagged [optimization]

Use this tag for any use of optimization within statistics.

2,783 questions
Filter by
Sorted by
Tagged with
11 views

### Derivation of dual formulation of support vector regression

I'm trying to derive the dual formulation of epsilon-insensitive support vector regression. I think my derivation is correct, but I can't match it up to a result for the dual that I've seen given in ...
• 225
33 views

• 68.3k
7 views

### How to find a linear decision boundary of a linearly separable problem with unlimited class evaluations?

I have a binary classification problem, where my goal is to find a linear decision boundary (which I assume exists). The context of the problem is that I have an iterative optimization process, where ...
15 views

### Optimisation of Polynomial Fittting Process

I have built a multitvariate log link GLM model and I want to fit polynomials to some of the numerical variates (i.e. fit polynomials of order 1,2,3 etc to the relativities of the model). However, I ...
1 vote
72 views

### Non-linear regression with very noisy data with nls() in R

I am trying to fit noisy data to a specific model with two parameters which I would like to estimate. Unfortunately, the model fit is just terrible with added noise. Is there anything I can do to ...
• 11
1 vote
33 views

### Optimizing objective with two variables multiplied with each other? [closed]

Let's say you want to optimize the following objective function: $$\min_{a,b} \Vert a + ab + b - W \Vert_2^2$$ where $a \in \mathbb{R}^{m,n}$ and $b \in \mathbb{R}^{n,p}$ are the learnable matrices, ...
• 121
18 views

• 33
166 views

### Scenario where minimizing 0-1 loss is different than minimizing hinge loss

Suppose we're using linear predictors. I'm trying to conceptually understand how minimizing hinge loss and 0-1 loss aren't necessarily the same. For instance I was told that one can choose a set of ...
68 views

### Why can the method of moments be expressed as a minimization problem?

Generalized method of moments (GMM) estimation seems to be called generalized method of moments because the standard method of moments (MoM) is a special case, following the following logic. MoM is ...
• 64.2k
37 views

in machine learning mastery website, i don't understand this paragraph. P(A|B) = P(B|A) * P(A) / P(B) We can simplify this calculation by removing the normalizing value of P(B) and describe the ...
66 views

### Advice on Gaussian Process Classifier optimisation best practises? [duplicate]

Hyperparameter Range Determination: My main challenge is in setting effective ranges for hyperparameters such as length_scale, noise_level, and sigma_0. Currently, for length_scale, I've used the ...
34 views

### Training loss reach to zero, then suddenly increases, then decreases to zero

I get the following loss behavior when training multilayer perceptron with mean squared error loss on some synthetic data using default Adam with default learning. (I am working on 1 demention data) I ...
23 views

### Transforming discrete optimisation problem into continuous optimisation problem

In Sparse Hilbert-Schmidt Independence Criterion Regression (Poignard and Yamada, AISTATS 2020), the authors consider a way to perform feature selection by taking the subset of features that maximises ...
• 51
34 views

### Is there a (lower) limit/minimum for learning rate values?

I'm building a model for traffic prediction with ConvLSTM and A3T-GCN cells. Since the input data is highly complex and the model is relatively big, I can only load ...
• 101
1 vote