# What Is the Loss (Objective) Function for Linear Discriminant Analysis (LDA)?

As many algorithms can be viewed as optimization problems through the Loss function, I was wondering if such a loss function existed for LDA (linear classification). And if yes, what would it be ?

• For SVM: $L_{Hinge}(y,x,w)=max(0,1-yw^tx),y\in\{-1,1\}$
• For Logistic Regression: $L_{Log}(y,x,w) =log(1+e^{-yw^tx}), ,y\in \{-1,1\}$,
• For Perceptron: $L_{Perceptron}(y,x,w)=max(0,-yw^tx),y\in \{-1,1\}$

where $x$ stand for the feature, $y$ the label and $w$ the parameter of the hyperplane we have to find.

Edit: Thanks a lot for your help but i don't know if i am clear enough. I am looking for the loss for 1 instance as if i were about to implement a stochastig gradient descent. Perhaps i miss something among your answers...

• the log likelihood – hxd1011 May 25 '18 at 18:10
• @hxd1011 negative log likelihood :-) – Łukasz Grad May 25 '18 at 18:46
• @ŁukaszGrad, Could you write the explicit form? – Royi May 25 '18 at 19:08
• – Royi May 25 '18 at 19:30
• For binary $y$, LDA is equivalent to linear regression of $y$ on $X$. So the loss function is simply squared error. CC @Royi. – amoeba May 25 '18 at 19:59