Stack Exchange Network

Stack Exchange network consists of 174 Q&A communities including Stack Overflow, the largest, most trusted online community for developers to learn, share their knowledge, and build their careers.

Visit Stack Exchange

Questions tagged [discriminant-analysis]

Linear Discriminant Analysis (LDA) is a dimensionality reduction and classification method. It finds low-dimensional subspace with the strongest class separation and uses it to perform classification. Use this tag for quadratic DA (QDA) too.

0
votes
0answers
12 views

What does “sphering and then classify to the closest centroid” mean?

What does "sphering and then classify to the closest centroid" mean? In linear discriminant classification. I understand that sphering or whitening is some kind of variance reduction and leads to a "...
0
votes
0answers
16 views

Linear Discriminat analysis in R [on hold]

I am trying to implement LDA in R using this article after computing the between-class matrix SB and within-class matrix SW how can I compute the discriminant functions and the decision boundary?
0
votes
0answers
6 views

How does one formulate a discriminant function that separates classes? [on hold]

How does one formulate a discriminant function that separates classes? Like how does one know, where to place it?
0
votes
0answers
18 views

A review of the linear and quadratic discriminant analyses (LDA-QDA) [closed]

I am asking if there is a review of LDA and QDA. Also, is there any paper sue Box's M test to test the heterogeneity of covariance matrices and they found that statically significant (which supposed ...
0
votes
0answers
9 views

singular within scatter matrix and non singular total scatter matrix

When is the scatter matrix in linear discriminant analysis singular although total scatter matrix is non singular ? on which conditions this happens? Or can you introduce me a book or paper to read ...
1
vote
0answers
19 views

Why the objective function in Fisher Discriminant analysis?

I know FDA wants to find some linear combination $z = W^T x$ so that the projected data has maximum between-class covariance and minimum within-class covariance. The first thing that came to my mind ...
0
votes
0answers
21 views

Proof of robustness of ``Linear Discriminant Analysis"

Assume one has been given $N$ data points in $\mathbb{R}^{d_1}$ each of which comes with a label from some set $\{1,\ldots,q\}$. Now I guess the claim is that doing (Linear Discriminant Analysis) LDA ...
1
vote
0answers
47 views

MASS{lda} plot in R [closed]

I'm reading the Introduction to statistical learning with R currently, but I blocked through a Lab about Discriminant analysis. So the thing is that we trying to fit a linear discriminant analysis ...
1
vote
0answers
18 views

Linear discriminant analysis and logistic regression

I have found in the script of the Machine Learning lecture CS229 by Andrew Ng at Stanford University, that he claims that (at least in the case of only two classes) the posterior of the linear ...
0
votes
0answers
24 views

Intuition on Linear Discriminant Analysis and Z-Test/T-Test

I am trying to get into Linear Discriminant Analysis without getting too much into linear algebra and also having a lack of geometric fantasy. To obtain an intuition of whats going on, I just want to ...
0
votes
0answers
8 views

optimality of LDA dimensionality reduction

typically when you do dimensionality reduction using LDA, you select $n_{class}-1$ vectors with largest eigenvalues as discriminants given the fact that you only need $n-1$ dimensions to classify $n$ ...
0
votes
0answers
21 views

Proving proportionality between LDA and simple linear regression coefficients

For a model with a single regressor $X$ and a response $Y$ of two classes ( $y=n_k/n$ if $x$ is of class $k$, $k=1,2$), the linear discriminant will be $$\delta(x)=x'\Sigma^{-1}\hat\mu_k -\frac{1}{2}\...
2
votes
0answers
34 views

Why Linear Discriminant analysis in MATLAB and R are producing different results

I am applying manova and lda to my data 12 samples (6 groups with 2 samples in each) and 6 measurements. I used lda form MAS5 package in R and different LDA functions in MATLAB. They all gave me ...
2
votes
1answer
44 views

Linear discriminant analysis with $p\gg n$

I am studying Linear Discriminant Analysis (LDA). According to the formula for LDA, we are supposed to get the inverse of within group covariance. However, if $p\gg n$ (i.e., the dimension is much ...
4
votes
1answer
75 views

Why is Linear Discriminant Analysis a linear classifier?

Question As the title suggests, I am wondering why we consider LDA a linear classifier. More specifically, I would like to know why LDA is considered a linear classifier in the following case: ...
5
votes
2answers
304 views

Why with two classes, LDA gives only one dimension?

I am working with dimensionality reduction algorithms. Linear Discriminant Analysis (LDA) is a supervised algorithm that takes into account the class label (which is not the case of PCA for example). ...
0
votes
0answers
12 views

Find the best “pair” of features to predict binary response with lda

Let's take the mtcars dataset: I want to find the 2 best features to predict vs 0 or ...
0
votes
0answers
36 views

How does SPSS “standardize” variables for discriminant analysis?

We are trying to reproduce the outcome of a discriminant analysis done in SPSS and run into a mystery. This link: https://stats.idre.ucla.edu/spss/output/discriminant-analysis/ explains that ...
1
vote
0answers
19 views

Linear Discriminant Analysis vs Quadratic Discriminant Analysis [duplicate]

I'm looking to find out peoples thoughts on LDA and QDA. When is it better to use one than the other? Also advantages and disadvantages of using both methods for classification purposes.
0
votes
1answer
21 views

Which analysis to use to discriminate morphometrics measurements from different species from 2 different environment?

So I have a dataset of measurements (lengths, surface areas, volumes...) from 3 species from 2 different environments, with 3 individuals per species. Can be summarised like that: ...
0
votes
0answers
14 views

Discriminant analysis with non-independent observations

I am using an experimental point-of-care test to detect infection of skin ulcers. The test measures X different features, and I am using the R "caret" package to perform a partial least square ...
0
votes
0answers
37 views

LDA - direction which maximizes class separation

As far as I understood - at least form a very raw conceptual point of view, LDA (Linear Discriminant Analysis), when used as a dimensional reduction technique, does two things (I'll stick to the 2-...
1
vote
0answers
14 views

Non-Negative and Sparse LDA?

I am wondering if the concept of Non-Negative and Sparse Linear Discriminant Analysis exists? I recently found an algorithm for Non-Negative and Sparse Principal Component Analysis (on which the PC ...
0
votes
0answers
34 views

Why LDA objective function is non convex?

I have read an article. Now it is my question, Why below optimization problem is nonconvex(it is the LDA objective function)? And when we find the dual solution what does that mean for the above ...
0
votes
0answers
17 views

ٌWhy we use SVD in Penalized Discriminant Analysis?

I have read Penalized Discriminant Analysis article (Hastie and Tibshirani). Actually, I am new in this field and I have a couple of questions about the article. There are 3 criterions which were ...
3
votes
0answers
89 views

Rayleigh quotient, traces and LDA optimization problem

I've been working about Linear Discriminant Analysis the last weeks, and after reading many articles, I see some aspects of this problem not very clear. The LDA optimization problem is formulated by ...
3
votes
1answer
120 views

Autoencoders and Collaborative Filtering: Is one network per training sample really necessary?

I have been researching on how to apply neural networks to recommender systems and have come across this paper (AutoRec: Autoencoders Meet Collaborative Filtering by Sedhain et al.) where they model ...
4
votes
2answers
78 views

Derivation of $S_W^{-1} S_B$ during the calculation of LDA

I try to reason the computations during the search for the optimal weight vector $w$ during the calculations of LDA. Therefore I use several text books like: Kuhn, M. and Johnson, K. (2013) Applied ...
2
votes
0answers
38 views

Why are the discriminant axes in linear discriminant analysis (LDA) not orthogonal?

This may be a quite silly question and please correct me if I'm wrong. The discriminants (discriminant axes) are essentially eigenvectors of $\mathrm{Cov}_\mathrm{within}^{-1} \mathrm{Cov}_\mathrm{...
3
votes
1answer
104 views

Looking for good, recent examples of Discriminant Analysis (Linear, Quadratic or else) in applied research

We have looked at LDA/QDA several times during my stats masters coursework, but I'm not convinced that it's due to the usefulness of the techniques more than my school being stuck with a 20-year old ...
0
votes
1answer
73 views

Using chi-squared or binomial for triangle test of discrimination

Regarding beer taste testing. The wonderful people at Brulosophy do many experiments with beer, changing brew and fermentation methods and then using triangle tests to see whether there's a difference....
1
vote
1answer
59 views

Why does Fisher use covariance when only variance is needed?

With reference to the following image from here: (can not inline it due to unsupported format) https://wikimedia.org/api/rest_v1/media/math/render/svg/9af8aa035642689bb2004047416b069a15406447 If we ...
2
votes
0answers
64 views

Contour algorithm - Classification plot

As Amoeba has highlighted in his answer, the authors of the book elements of statistical learning make use of a contour algorithm to produce visual plots of classification algorithms. For example, ...
3
votes
1answer
68 views

Fisher LDA - What is the difference between a discriminant function and a linear decision boundary?

I am studying Fisher LDA, the case where there are K=2 classes of data. It is my understanding that Fisher LDA looks for the 1-dimensional space onto which the data should be projected in order to ...
0
votes
0answers
28 views

Best statistical indicator to describe performance

Let's say I have several ingredients, each one described by two vectors : The first one is the concentration of the ingredient in the recipe (from 0 to 1000) The second one is a "liking" indicator ...
1
vote
1answer
330 views

What Is the Loss (Objective) Function for Linear Discriminant Analysis (LDA)?

As many algorithms can be viewed as optimization problems through the Loss function, I was wondering if such a loss function existed for LDA (linear classification). And if yes, what would it be ? I ...
1
vote
1answer
25 views

Multiple measurements within one feature

From US health insurance data, I have 300k prescriptions for an injected biologic from many patients . For each prescription there's a quantity variable, described by the data vendor as "The number ...
0
votes
0answers
43 views

Discriminant analysis sample size?

I have a data frame, where each item (animal) is characterized by 6 dependent variables (length of it's trait by mm). For each of them several grouping variables are given (Now I speak only about ...
1
vote
0answers
18 views

Classification of very large number of samples using LDA

I have a dataset with tens of thousands of samples and only 7 features. I want to use linear discriminant analysis to classify these samples into 2 classes. The ...
2
votes
1answer
559 views

How to find and draw the Bayes decision boundary in LDA (2 classes)?

Can anyone point me (or even show me if that's allowed here) how to calculate and draw the boundary line in LDA when we have only 2 classes? In the example I'm trying to do (in a paper), the ...
0
votes
0answers
13 views

QDA homogeneity of covariances

I am currently studying Quadratic Discriminant Analysis and came across the following query. In QDA the assumption that all covariances must be equal does not apply but what if all covariances are ...
0
votes
0answers
35 views

LDA for hyperspectral image band selection

I am trying to use LDA implemented in SPSS Statistics software to reduce dimensionality of hyperspectral data that I will classify in the next step. I have found many sources that LDA is commonly used ...
0
votes
1answer
52 views

Linear Discriminant Analysis with p>1 How does X transpose Sigma can get multiplied

I'm reading Bishop books on Statistical Learning, and ran into the Gaussian density function for LDA when p > 1. $$P(x|k)=\frac{1}{\sqrt{(2\pi)^p|\boldsymbol\Sigma|}} \exp\left(-\frac{1}{2}({x'}-{\...
0
votes
0answers
44 views

How to transform highly right skewed data with zeros to normal for LDA classification?

I have a dataset that I would like to perform classification using Linear discriminent Analysis , QDA, and Logistic Regrssion on, but I am having some trouble with it. my response varible is a ...
1
vote
1answer
150 views

Combining PLS-DA with PCA dimension reduction

I am implementing the PLS-DA method presented here on a data set and I am trying to understand the procedure; and whether there is anything conceptually wrong in my steps. I start with $188 \times ...
0
votes
1answer
160 views

PLS-DA for varible selection and then LDA for classification

I have been trying to do classification of my hyperspectral data. The variable selection were done with variable importance in projection in pls-da. The selected variables were then used for ...
0
votes
0answers
31 views

Scaling of Linear Discriminants

What is the meaning of "a matrix which transforms observations to discriminant functions, normalized so that within groups covariance matrix is spherical" from ...
2
votes
0answers
231 views

Scaling of linear discriminant from lda in MASS

I was attempting to replicate in R an example my instructor used in class from SPSS. However, it looks like the lda$scaling value function from ...
0
votes
0answers
18 views

total variability multivariate case

I need to prove the next equality: Show that $$\sum_{i,j=1}^n(X_i-X_j)(X_i-X_j)^T=2nSSP_{TOTAL}$$ where $SSP_{TOTAL}=\sum_{i=1}^n(X_i-\bar X)(X_i-\bar X)^T$ First, I am not clear because I was ...
1
vote
0answers
34 views

Is the Fisher Linear Discriminant classifier an ERM?

Can the Fisher Linear Discriminant classifier be considered an Empirical Risk Mininisation (ERM) classifier? I know of a formulation as 2 Gaussians with identical covariance, and another that ...