# Questions tagged [discriminant-analysis]

Linear Discriminant Analysis (LDA) is a dimensionality reduction and classification method. It finds low-dimensional subspace with the strongest class separation and uses it to perform classification. Use this tag for quadratic DA (QDA) too.

372 questions
18 views

### Classification with ONLY categorical data

Suppose I have a table with some factor characteristics of some plants. For instance, petal color, pollen color, and so on. What is the best way to classify that data? Is it feasible to use some of ...
10 views

### Searching for a python function that performs discriminant analysis using Mahalanobis distances, like in Matlab? [closed]

I'm trying to classify vectors using discriminant analysis which uses Mahalanobis distances with stratified covariance estimates. The paper I'm using as a reference has used a matlab function to ...
4 views

### cluster uncertainty, silhoutte coeffs and classification accuracy from DFA. How are these related?

I am slightly confused by some of the results I am getting from model based cluster analyses (mclust in R), calculating silhouette coefficients (silhouettes in R based on euclidian distances) for ...
16 views

### Fisher's Linear Discriminant Analysis

I have a dataset with two classes and I want to apply Fisher's Linear Discriminant Analysis. To train the model, in what order do I need to compute the following:The within-class scatter,The sample ...
6 views

### Ranking the conditional probability of classification study (PLSDA) to get better accuracy

I am building a model to predict the probability of an animal to be poor or good, using PLSDA. The sensitivity and specificity are 76% and 56%, respectively. Let's ignore the sensitivity for now, ...
12 views

### How can I classify a person's score according to my discriminant variable, in a discriminant function? (multiple discriminant analysis) [closed]

I'm doing a research on 5 types of entrepreneurship (dependent variable) and personality traits (5 independent variables). As you can tell, I did a discriminant analysis as a "plus" for my prior ...
26 views

### Compute true positive rate from only predicted probability and actual probability (Binary LDA classifier)

I'm sort of stuck on this question and I can't find a similar problem online. Consider the following to be given: 1.input x is 1D, output y is binary {0,1} 2.marginal probability of y is $\pi_y=P(y)$ ...
25 views

### LDA Fit using Caret does not give Standard Deviation

I am following the steps outlined in this tutorial. I have followed along and running into an issue at step 5.3. The output of the LDA model gives me all the expected information, except the Accuracy ...
19 views

### LDA attribute similar to fitted values from glm

I am comparing the results of LDA vs logistic regression as an exercise in understanding their differences and similarities. When fitting a glm model ...
30 views

### Linear discrimant analysis: Proof MLE covariance matrix is biased

I know that the MLE of the covariance matrix for LDA is: $$\Sigma_{MLE}=\dfrac{1}{N}\sum_{k=1}^K\sum_{\substack{i=1}{g_i=k}}^N(x_i-\hat{\mu}_{k})(x_i-\hat{\mu}_{k})^T.$$ How can I proof that this is a ...
43 views

### How do you normalise a histogram with two peaks?

I have a histogram that looks as such and I want to use it as part of a Linear Discriminant Analysis but the lda requires its variables to have a normal distribution. What kind of transformation ...
18 views

### proportion of total dataset variance explained by prior grouping in DAPC

I have read several treads but it still unclear to me how to estimate the proportion of variance that can be explained by a prior grouping when "Discriminant Component Analysis" (DAPC) is performed. ...
37 views

### Common covariance matrix explanation (LDA and QDA)

I'm looking for a layman's explanation of the "common covariance matrix" assumption in LDA because I don't think I understand it. I understand that a common covariance matrix (as assumed in LDA for ...
42 views

### The Linear Discriminant Analysis Rule

Given there are two classes A and B and the prior probability of belonging to $A = Na/N$ and $B = Nb/N$, I want to show that the linear discriminant analysis rule classifies an observation x to ...
28 views

### Linear discriminant scores using caret

I'm using the caret package in R to undertake an LDA. I'm having problems trying to extract the linear discriminant scores once I've used predict. The model is ... ...
17 views

### Linear Discriminant Analysis vocabulary question

I am doing a descriptive LDA on a dataset with two (known, easily separable) classes and many features (and many more observations). I intend to use the latent variable values as a dimensionally-...
28 views

### Linear discriminant analysis against quadratic discriminant analysis behavior in R

I am using linear discriminant analysis (LDA) and quadratic discriminant analysis (QDA) in R. I am working on a small data set of 4 observations and two variables <...
44 views

### What are some good robust loss functions for binary classification using LDA?

I am doing a project where I use LDA for binary classification. I want to know how it performs when there are outliers. What are some good robust loss functions for binary classification?
70 views

### Estimating the true error rate from the optimistic resubstitution (or apparent) error rate of a PLS-DA model

In short, I'm trying to calculate the 'Upper bound 3' [Ref, p.4] of the true error rate for my partial least squares regression model (PLS-DA), separating two classes A and B of a sample set. Let me ...
39 views

### Is LDA just selecting the minimum Mahalanobis distance?

I have a question regarding Linear Discriminant Analysis (LDA). I know that LDA chooses the coefficients of a linear model, which maximize the separability of classes - that is the ratio of "between-...
19 views

### Can we interpret the value of a Discriminant Function as a probability?

Suppose I have fit a classification model through Genetic Programming. The output is a symbolic expression that is a function of the covariates, and that associates to each data point a class, such ...
85 views

### LDA - solving singularity problem of within classes matrix

I would like to solve the problem in LDA where the within classes matrix is singular if the number of samples is lower than the number of dimensions (which is true in my case, used on images of faces)....
12 views

### Identifying variables which differentiate 2 groups the most

I have 2 groups of patients (having and not having a disease A), respectively 41 and 19 patients. Set of about 25 different parameters was measured at those patients (some continuous variables and a ...
20 views

### How to find orthogonal projection of a d-dimensional point, d>2 on Fisher's discriminant?

We know that the number of Fisher's discriminant to find for classifying data in given dimension d into output classes K, is min(K-1,d) Using the iris dataset as an example, we know that d=4 (petal ...
37 views

### What does “sphering and then classify to the closest centroid” mean?

What does "sphering and then classify to the closest centroid" mean? In linear discriminant classification. I understand that sphering or whitening is some kind of variance reduction and leads to a "...
25 views

### singular within scatter matrix and non singular total scatter matrix

When is the scatter matrix in linear discriminant analysis singular although total scatter matrix is non singular ? on which conditions this happens? Or can you introduce me a book or paper to read ...
30 views

### Why the objective function in Fisher Discriminant analysis?

I know FDA wants to find some linear combination $z = W^T x$ so that the projected data has maximum between-class covariance and minimum within-class covariance. The first thing that came to my mind ...
31 views

### Proof of robustness of Linear Discriminant Analysis"

Assume one has been given $N$ data points in $\mathbb{R}^{d_1}$ each of which comes with a label from some set $\{1,\ldots,q\}$. Now I guess the claim is that doing (Linear Discriminant Analysis) LDA ...
158 views

### MASS{lda} plot in R [closed]

I'm reading the Introduction to statistical learning with R currently, but I blocked through a Lab about Discriminant analysis. So the thing is that we trying to fit a linear discriminant analysis ...
178 views

### Linear discriminant analysis and logistic regression

I have found in the script of the Machine Learning lecture CS229 by Andrew Ng at Stanford University, that he claims that (at least in the case of only two classes) the posterior of the linear ...
14 views

### optimality of LDA dimensionality reduction

typically when you do dimensionality reduction using LDA, you select $n_{class}-1$ vectors with largest eigenvalues as discriminants given the fact that you only need $n-1$ dimensions to classify $n$ ...
57 views

### Why Linear Discriminant analysis in MATLAB and R are producing different results

I am applying manova and lda to my data 12 samples (6 groups with 2 samples in each) and 6 measurements. I used lda form MAS5 package in R and different LDA functions in MATLAB. They all gave me ...
77 views

### Linear discriminant analysis with $p\gg n$

I am studying Linear Discriminant Analysis (LDA). According to the formula for LDA, we are supposed to get the inverse of within group covariance. However, if $p\gg n$ (i.e., the dimension is much ...
223 views

### Why is Linear Discriminant Analysis a linear classifier?

Question As the title suggests, I am wondering why we consider LDA a linear classifier. More specifically, I would like to know why LDA is considered a linear classifier in the following case: ...
938 views

### Why with two classes, LDA gives only one dimension?

I am working with dimensionality reduction algorithms. Linear Discriminant Analysis (LDA) is a supervised algorithm that takes into account the class label (which is not the case of PCA for example). ...
15 views

### Find the best “pair” of features to predict binary response with lda

Let's take the mtcars dataset: I want to find the 2 best features to predict vs 0 or ...
52 views

### How does SPSS “standardize” variables for discriminant analysis?

We are trying to reproduce the outcome of a discriminant analysis done in SPSS and run into a mystery. This link: https://stats.idre.ucla.edu/spss/output/discriminant-analysis/ explains that ...
66 views

### Linear Discriminant Analysis vs Quadratic Discriminant Analysis [duplicate]

I'm looking to find out peoples thoughts on LDA and QDA. When is it better to use one than the other? Also advantages and disadvantages of using both methods for classification purposes.
25 views

### Which analysis to use to discriminate morphometrics measurements from different species from 2 different environment?

So I have a dataset of measurements (lengths, surface areas, volumes...) from 3 species from 2 different environments, with 3 individuals per species. Can be summarised like that: ...
47 views

### LDA - direction which maximizes class separation

As far as I understood - at least form a very raw conceptual point of view, LDA (Linear Discriminant Analysis), when used as a dimensional reduction technique, does two things (I'll stick to the 2-...
23 views

### Non-Negative and Sparse LDA?

I am wondering if the concept of Non-Negative and Sparse Linear Discriminant Analysis exists? I recently found an algorithm for Non-Negative and Sparse Principal Component Analysis (on which the PC ...
68 views

### Why LDA objective function is non convex?

I have read an article. Now it is my question, Why below optimization problem is nonconvex(it is the LDA objective function)? And when we find the dual solution what does that mean for the above ...
43 views

### ٌWhy we use SVD in Penalized Discriminant Analysis?

I have read Penalized Discriminant Analysis article (Hastie and Tibshirani). Actually, I am new in this field and I have a couple of questions about the article. There are 3 criterions which were ...
139 views

### Rayleigh quotient, traces and LDA optimization problem

I've been working about Linear Discriminant Analysis the last weeks, and after reading many articles, I see some aspects of this problem not very clear. The LDA optimization problem is formulated by ...
190 views

### Autoencoders and Collaborative Filtering: Is one network per training sample really necessary?

I have been researching on how to apply neural networks to recommender systems and have come across this paper (AutoRec: Autoencoders Meet Collaborative Filtering by Sedhain et al.) where they model ...
97 views

### Derivation of $S_W^{-1} S_B$ during the calculation of LDA

I try to reason the computations during the search for the optimal weight vector $w$ during the calculations of LDA. Therefore I use several text books like: Kuhn, M. and Johnson, K. (2013) Applied ...
84 views

### Why are the discriminant axes in linear discriminant analysis (LDA) not orthogonal?

This may be a quite silly question and please correct me if I'm wrong. The discriminants (discriminant axes) are essentially eigenvectors of \$\mathrm{Cov}_\mathrm{within}^{-1} \mathrm{Cov}_\mathrm{...
127 views

### Looking for good, recent examples of Discriminant Analysis (Linear, Quadratic or else) in applied research

We have looked at LDA/QDA several times during my stats masters coursework, but I'm not convinced that it's due to the usefulness of the techniques more than my school being stuck with a 20-year old ...