What Is the Loss (Objective) Function for Linear Discriminant Analysis (LDA)?

Question

As many algorithms can be viewed as optimization problems through the Loss function, I was wondering if such a loss function existed for LDA (linear classification). And if yes, what would it be ?

I already know these:

• For SVM: $L_{Hinge}(y,x,w)=max(0,1-yw^tx),y\in\{-1,1\}$
• For Logistic Regression: $L_{Log}(y,x,w) =log(1+e^{-yw^tx}), ,y\in \{-1,1\}$,
• For Perceptron: $L_{Perceptron}(y,x,w)=max(0,-yw^tx),y\in \{-1,1\}$

where $x$ stand for the feature, $y$ the label and $w$ the parameter of the hyperplane we have to find.

Edit: Thanks a lot for your help but i don't know if i am clear enough. I am looking for the loss for 1 instance as if i were about to implement a stochastig gradient descent. Perhaps i miss something among your answers...

Answer 1

LDA is also called Fisher’s linear discriminant. I refer you to page 186 of book “Pattern recognition and machine learning” by Christopher Bishop. The objective function that you are looking for is called Fisher’s criterion J(w) and is formulated in page 188 of the book. The Fisher criterion is defined to be the ratio of the between-class variance to the within-class variance.