# Questions tagged [lagrange-multipliers]

The method of Lagrange multipliers finds critical points (including maxima and minima) of a differentiable function subject to differentiable constraints.

66 questions
Filter by
Sorted by
Tagged with
15 views

### Trying to understand lagrange multiplier and lads

I’m trying to do the lagrange multiplier test manually accordig to the below text provided by my supervisor. I do not fully understand what he means. Could anyone maybe describe it in R syntax? He ...
34 views

### deriving the optimal distribution

Let the input variable $X \in \mathcal{X}$ and the target variable $Y \in \mathcal{Y}$. For a fixed hypothesis $h \in \mathcal{H}$ I want to solve \min_{p(X,Y)} \int_{\mathcal{X}}\...
• 27
1 vote
37 views

### incorporating a distribution constraint in a minimisation objective

For a given (convex) hypothesis $h \in \mathcal{H}$, and the variables $X \in \mathcal{X}$ and $Y \in \mathcal{Y}$ I have the following optimisation problem: \min_{p(X,Y)} \int_{\...
• 27
1 vote
28 views

### Derivation of the SVM regression problem

EDIT: The terms at the end will not get cancelled as they involve different variables: $\alpha$ and $\alpha^*$, etc. Now the post is corrected. I will leave this question here in case someone is as ...
• 259
19 views

### What's a Heterogenous time series and how does Lagrange multiplier test statistic relate to it?

I have two time series with me. Running analysis with fb KATS time series Analysis module, I get below values for heterogeneity. ...
37 views

### SVM: How to derive the KKT condition with soft margin term is quadratic?

I was reading the Introdunction to Data Mining (2013) when I came across this in section 5.5: $$f(\textbf{w})=\frac{||\textbf{w}||^2}{2}+C\left(\sum^N_{i=1}\xi_i \right)^k$$ I found similar ...
43 views

### Why $\gamma$ in regularization term of XGBoost is defined as minimum loss reduction (not minimum squared loss reduction) and not substracted?

From the source https://xgboost.readthedocs.io/en/stable/tutorials/model.html I guess that the mean-squared error is optimized subjected to a constraint of minimum loss reduction. It appears like ...
• 11
30 views

### Primal Problem of SVM

The primal problem of SVM is denoted as below. $$min_{w,b}\left(\phi \:\left(w\right)\right)=min_{w,b}\left(\frac{1}{2}w^Tw\right)$$ Subject to $$y_n\left(w^Tx_n+b\right)\ge 1,\:n=1,2,3,...,l$$ And If ...
39 views

### KKT Conditions for thresholds?

My main question is that when I use Lagrange Multipliers/KKT conditions to perform optimization with threshold constraints, I seem to get contradictory FOC. Here is a characteristic example: take an ...
• 23
23 views

### Doubt about Lagrange Multiplier Statistics for q exclusions restrictions

For obtaining the Lagrange Multiplier Statistics I follow this steps (Wooldridge 2019, Introductory Econometrics): Regress $y$ on the restricted set of independent variables and consider the ...
• 69
23 views

### Lagrangian multiplier with an equality constraint and an inequality constraint

I want to compute the maximum of a function $f(x)$ with an equality constraint, $g(x) = 0$ and an inequality constraint, $h(x)\geq 0$. I would know what to do with only an equality constraint, or ...
1 vote
137 views

• 996
1 vote
440 views

### How to derive the $\ nR^2$ form of the LM test statistic?

Several lagrange multiplier (LM) test statistics =$\ nR^2$~$\chi^2$, where the LM test statistic is generated from regressing the square of residuals on some function in an auxiliary equation. The ...
• 63
35 views

• 41
1 vote
14 views

### Score test using restricted mle

I am doing a study on Rao score test and still confused about the score vector. How am I supposed to calculate the score vector? Can I just use (ˆθn - θn) for the score vector? As I understood that ...
• 11
1 vote
32 views

### Rewrite $\frac{1}{2}||x-u||_2^2$ subject to $||x||_1\le c$ to lagrangian form with multiplier $\lambda \ge 0$

Rewrite $\frac{1}{2}||x-u||_2^2$ subject to $||x||_1\le c$ to lagrangian form with multiplier $\lambda \ge 0$ So I'm pretty new to converting constraint functions to Lagrangian form, but I read that ...
• 640
46 views

### Non-negative Constraints in Soft-Margin SVM Lagrange Equation

I was reading the A Tutorial on Support Vector Machines for Pattern Recognition as a supplemental for my Intro to ML class and I wasn't sure why $a_i \geq 0$ and $\lambda_i \geq 0$ cannot be ...
• 1
95 views

189 views

### Maximum Entropy Discrete Distribution

In Pattern Recognition and Machine Learning the author uses Lagrange multipliers to find the discrete distribution with maximum entropy. Entropy is defined by; $$H=-\sum_i p(x_i)\ln(p(x_i))$$ and the ...
1 vote
38 views

### What is $\alpha$ in SVM Lagrangian fucntions

Is it a column matrix or a row matrix? I saw on a CMU slide that $\alpha$ is defined as $\alpha=(\alpha_1, \alpha_2, ..., \alpha_n)^T>=0$? I dont see much of a formal definition for the exact form ...
1 vote
43 views

### Convert the following expression w.r.t to the whole dataset instead of element of the dataset?

I am in the process of expressing the w in LSSVM with data points and constants. After I resolve the KKT conditions for the LSSVM I got $$w = \sum ^N _{i=1} \alpha_i x_i$$ Is it possible to convert ...
1 vote
166 views

### Minimize $f(A,B)$ s.t. $\text{exp}(A)^T \text{exp}(B)=J_K$

I have a function $f(A,B)$ that maps a pair of two (tall) matrices, $A$ and $B$, to a scalar cost that I want to minimize. $A$ and $B$ both have $K$ columns. I also want to impose a set of equality ...
• 6,600
1 vote
19 views

• 447
1 vote
3k views

### Calculating the value of $b^{*}$ in an SVM

In Andrew Ng's notes on SVMs, he claims that once we solve the dual problem and get $\alpha^*$ we can calculate $w^*$ and consequently calculate $b^*$ from the primal to get equation (11) (see notes) ...
• 63
28 views

### If the Lagrangian formulation also has constraints, what is then the simplification?

Consider the following constrained optimization problem. $$\begin{array}{ll} \text{minimize} & f(w)\\ \text{subject to} & g_{i}(w) \leq 0\\ & h_{j}(w) = 0\end{array}$$ The equivalent ...
• 141
1 vote
74 views

1k views