New answers tagged

0 votes

Maxima in the dual of hard margin and soft margin SVMs?

The soft margin SVM has a maximum, provided that the feasibility region is not empty. By Weierstrass' I and II theorems, continuous functions have minima and maxima over compact sets, such as the ...
Alex Teush's user avatar
1 vote

Newton's method and the Hessian for Softmax Logistic Regression

Here's one issue: You have $\mathbf{w}\in \mathbb{R}^{n_\text{features} n_\text{classes}}$ instead of $\mathbf{w}\in \mathbb{R}^{n_\text{features} (n_\text{classes}-1)}$. One of the relative class ...
Sextus Empiricus's user avatar
3 votes
Accepted

Why does nlme::fdHess return NaNs when evaluated at 0?

Looking at the code it seems as though the basic problem here is that the function, by default, chooses the finite difference step size relative to the absolute value of the parameters: in the ...
Ben Bolker's user avatar
  • 40.2k
0 votes
Accepted

Censored Regression model

After replacing pnorm(z) with pnorm(xb/sigma), I find: ...
Stéphane Laurent's user avatar
0 votes

How do I obtain the primal and dual for the estimator $\min _\beta\left[\|\beta\|^2+\sum_{i=1}^n \xi_i^2\right]$ s.t. $\xi_i=y_i-h(x_i)^\top \beta$?

Per (i), the Lagrangian isn't defined by plugging in the equality constraints multiplied by Lagrange multipliers to the objective function. Given an optimization problem with equality constraints $\{...
Alex Teush's user avatar
1 vote

How do I obtain the primal and dual for the estimator $\min _\beta\left[\|\beta\|^2+\sum_{i=1}^n \xi_i^2\right]$ s.t. $\xi_i=y_i-h(x_i)^\top \beta$?

There's a few minor issues in the earlier parts -- I'll provide some hints. (i) If I recall correctly this is simply the Lagrangian function (i.e., the objective of the Wolfe dual). For an equality-...
chang_trenton's user avatar
4 votes

Example where the initial random state of a logistic regression matters?

The solution to OLS is not unique unless the design matrix is of full rank. If the matrix is rank deficient, there are actually several possible "solutions" using the pseudoinverse of $(X^{T}...
AdamO's user avatar
  • 59.4k
7 votes

Example where the initial random state of a logistic regression matters?

Both linear and logistic regression are convex optimisation problems and have same behaviour. If the 2nd derivative of the objective at the minimum is positive definite, then the minimum is unique, ...
seanv507's user avatar
  • 5,828

Top 50 recent answers are included