New answers tagged optimization
0
votes
Maxima in the dual of hard margin and soft margin SVMs?
The soft margin SVM has a maximum, provided that the feasibility region is not empty. By Weierstrass' I and II theorems, continuous functions have minima and maxima over compact sets, such as the ...
- 26
1
vote
Newton's method and the Hessian for Softmax Logistic Regression
Here's one issue:
You have $\mathbf{w}\in \mathbb{R}^{n_\text{features} n_\text{classes}}$
instead of $\mathbf{w}\in \mathbb{R}^{n_\text{features} (n_\text{classes}-1)}$. One of the relative class ...
- 68.3k
3
votes
Accepted
Why does nlme::fdHess return NaNs when evaluated at 0?
Looking at the code it seems as though the basic problem here is that the function, by default, chooses the finite difference step size relative to the absolute value of the parameters: in the ...
- 40.2k
0
votes
Accepted
0
votes
How do I obtain the primal and dual for the estimator $\min _\beta\left[\|\beta\|^2+\sum_{i=1}^n \xi_i^2\right]$ s.t. $\xi_i=y_i-h(x_i)^\top \beta$?
Per (i), the Lagrangian isn't defined by plugging in the equality constraints multiplied by Lagrange multipliers to the objective function.
Given an optimization problem with equality constraints $\{...
- 26
1
vote
How do I obtain the primal and dual for the estimator $\min _\beta\left[\|\beta\|^2+\sum_{i=1}^n \xi_i^2\right]$ s.t. $\xi_i=y_i-h(x_i)^\top \beta$?
There's a few minor issues in the earlier parts -- I'll provide some hints.
(i) If I recall correctly this is simply the Lagrangian function (i.e., the objective of the Wolfe dual). For an equality-...
- 1,533
4
votes
Example where the initial random state of a logistic regression matters?
The solution to OLS is not unique unless the design matrix is of full rank. If the matrix is rank deficient, there are actually several possible "solutions" using the pseudoinverse of $(X^{T}...
- 59.4k
7
votes
Example where the initial random state of a logistic regression matters?
Both linear and logistic regression are convex optimisation problems and have same behaviour.
If the 2nd derivative of the objective at the minimum is positive definite, then the minimum is unique, ...
- 5,828
Top 50 recent answers are included