Stack Exchange Network

Stack Exchange network consists of 175 Q&A communities including Stack Overflow, the largest, most trusted online community for developers to learn, share their knowledge, and build their careers.

Visit Stack Exchange

Questions tagged [discriminant-analysis]

Linear Discriminant Analysis (LDA) is a dimensionality reduction and classification method. It finds low-dimensional subspace with the strongest class separation and uses it to perform classification. Use this tag for quadratic DA (QDA) too.

0
votes
0answers
18 views

Classification with ONLY categorical data

Suppose I have a table with some factor characteristics of some plants. For instance, petal color, pollen color, and so on. What is the best way to classify that data? Is it feasible to use some of ...
0
votes
0answers
10 views

Searching for a python function that performs discriminant analysis using Mahalanobis distances, like in Matlab? [closed]

I'm trying to classify vectors using discriminant analysis which uses Mahalanobis distances with stratified covariance estimates. The paper I'm using as a reference has used a matlab function to ...
0
votes
0answers
4 views

cluster uncertainty, silhoutte coeffs and classification accuracy from DFA. How are these related?

I am slightly confused by some of the results I am getting from model based cluster analyses (mclust in R), calculating silhouette coefficients (silhouettes in R based on euclidian distances) for ...
0
votes
1answer
16 views

Fisher's Linear Discriminant Analysis

I have a dataset with two classes and I want to apply Fisher's Linear Discriminant Analysis. To train the model, in what order do I need to compute the following:The within-class scatter,The sample ...
0
votes
0answers
6 views

Ranking the conditional probability of classification study (PLSDA) to get better accuracy

I am building a model to predict the probability of an animal to be poor or good, using PLSDA. The sensitivity and specificity are 76% and 56%, respectively. Let's ignore the sensitivity for now, ...
1
vote
0answers
12 views

How can I classify a person's score according to my discriminant variable, in a discriminant function? (multiple discriminant analysis) [closed]

I'm doing a research on 5 types of entrepreneurship (dependent variable) and personality traits (5 independent variables). As you can tell, I did a discriminant analysis as a "plus" for my prior ...
0
votes
1answer
26 views

Compute true positive rate from only predicted probability and actual probability (Binary LDA classifier)

I'm sort of stuck on this question and I can't find a similar problem online. Consider the following to be given: 1.input x is 1D, output y is binary {0,1} 2.marginal probability of y is $\pi_y=P(y)$ ...
1
vote
0answers
25 views

LDA Fit using Caret does not give Standard Deviation

I am following the steps outlined in this tutorial. I have followed along and running into an issue at step 5.3. The output of the LDA model gives me all the expected information, except the Accuracy ...
0
votes
0answers
19 views

LDA attribute similar to fitted values from glm

I am comparing the results of LDA vs logistic regression as an exercise in understanding their differences and similarities. When fitting a glm model ...
0
votes
0answers
30 views

Linear discrimant analysis: Proof MLE covariance matrix is biased

I know that the MLE of the covariance matrix for LDA is: $$\Sigma_{MLE}=\dfrac{1}{N}\sum_{k=1}^K\sum_{\substack{i=1}{g_i=k}}^N(x_i-\hat{\mu}_{k})(x_i-\hat{\mu}_{k})^T.$$ How can I proof that this is a ...
0
votes
1answer
43 views

How do you normalise a histogram with two peaks?

I have a histogram that looks as such and I want to use it as part of a Linear Discriminant Analysis but the lda requires its variables to have a normal distribution. What kind of transformation ...
0
votes
0answers
18 views

proportion of total dataset variance explained by prior grouping in DAPC

I have read several treads but it still unclear to me how to estimate the proportion of variance that can be explained by a prior grouping when "Discriminant Component Analysis" (DAPC) is performed. ...
0
votes
0answers
37 views

Common covariance matrix explanation (LDA and QDA)

I'm looking for a layman's explanation of the "common covariance matrix" assumption in LDA because I don't think I understand it. I understand that a common covariance matrix (as assumed in LDA for ...
1
vote
0answers
42 views

The Linear Discriminant Analysis Rule

Given there are two classes A and B and the prior probability of belonging to $ A = Na/N $ and $B = Nb/N $, I want to show that the linear discriminant analysis rule classifies an observation x to ...
0
votes
0answers
28 views

Linear discriminant scores using caret

I'm using the caret package in R to undertake an LDA. I'm having problems trying to extract the linear discriminant scores once I've used predict. The model is ... ...
0
votes
1answer
17 views

Linear Discriminant Analysis vocabulary question

I am doing a descriptive LDA on a dataset with two (known, easily separable) classes and many features (and many more observations). I intend to use the latent variable values as a dimensionally-...
0
votes
0answers
28 views

Linear discriminant analysis against quadratic discriminant analysis behavior in R

I am using linear discriminant analysis (LDA) and quadratic discriminant analysis (QDA) in R. I am working on a small data set of 4 observations and two variables <...
1
vote
1answer
44 views

What are some good robust loss functions for binary classification using LDA?

I am doing a project where I use LDA for binary classification. I want to know how it performs when there are outliers. What are some good robust loss functions for binary classification?
0
votes
1answer
70 views

Estimating the true error rate from the optimistic resubstitution (or apparent) error rate of a PLS-DA model

In short, I'm trying to calculate the 'Upper bound 3' [Ref, p.4] of the true error rate for my partial least squares regression model (PLS-DA), separating two classes A and B of a sample set. Let me ...
1
vote
1answer
39 views

Is LDA just selecting the minimum Mahalanobis distance?

I have a question regarding Linear Discriminant Analysis (LDA). I know that LDA chooses the coefficients of a linear model, which maximize the separability of classes - that is the ratio of "between-...
0
votes
0answers
19 views

Can we interpret the value of a Discriminant Function as a probability?

Suppose I have fit a classification model through Genetic Programming. The output is a symbolic expression that is a function of the covariates, and that associates to each data point a class, such ...
0
votes
4answers
85 views

LDA - solving singularity problem of within classes matrix

I would like to solve the problem in LDA where the within classes matrix is singular if the number of samples is lower than the number of dimensions (which is true in my case, used on images of faces)....
0
votes
0answers
12 views

Identifying variables which differentiate 2 groups the most

I have 2 groups of patients (having and not having a disease A), respectively 41 and 19 patients. Set of about 25 different parameters was measured at those patients (some continuous variables and a ...
0
votes
0answers
20 views

How to find orthogonal projection of a d-dimensional point, d>2 on Fisher's discriminant?

We know that the number of Fisher's discriminant to find for classifying data in given dimension d into output classes K, is min(K-1,d) Using the iris dataset as an example, we know that d=4 (petal ...
0
votes
0answers
37 views

What does “sphering and then classify to the closest centroid” mean?

What does "sphering and then classify to the closest centroid" mean? In linear discriminant classification. I understand that sphering or whitening is some kind of variance reduction and leads to a "...
0
votes
0answers
25 views

singular within scatter matrix and non singular total scatter matrix

When is the scatter matrix in linear discriminant analysis singular although total scatter matrix is non singular ? on which conditions this happens? Or can you introduce me a book or paper to read ...
1
vote
0answers
30 views

Why the objective function in Fisher Discriminant analysis?

I know FDA wants to find some linear combination $z = W^T x$ so that the projected data has maximum between-class covariance and minimum within-class covariance. The first thing that came to my mind ...
0
votes
0answers
31 views

Proof of robustness of ``Linear Discriminant Analysis"

Assume one has been given $N$ data points in $\mathbb{R}^{d_1}$ each of which comes with a label from some set $\{1,\ldots,q\}$. Now I guess the claim is that doing (Linear Discriminant Analysis) LDA ...
2
votes
0answers
158 views

MASS{lda} plot in R [closed]

I'm reading the Introduction to statistical learning with R currently, but I blocked through a Lab about Discriminant analysis. So the thing is that we trying to fit a linear discriminant analysis ...
2
votes
1answer
178 views

Linear discriminant analysis and logistic regression

I have found in the script of the Machine Learning lecture CS229 by Andrew Ng at Stanford University, that he claims that (at least in the case of only two classes) the posterior of the linear ...
0
votes
0answers
14 views

optimality of LDA dimensionality reduction

typically when you do dimensionality reduction using LDA, you select $n_{class}-1$ vectors with largest eigenvalues as discriminants given the fact that you only need $n-1$ dimensions to classify $n$ ...
2
votes
0answers
57 views

Why Linear Discriminant analysis in MATLAB and R are producing different results

I am applying manova and lda to my data 12 samples (6 groups with 2 samples in each) and 6 measurements. I used lda form MAS5 package in R and different LDA functions in MATLAB. They all gave me ...
3
votes
1answer
77 views

Linear discriminant analysis with $p\gg n$

I am studying Linear Discriminant Analysis (LDA). According to the formula for LDA, we are supposed to get the inverse of within group covariance. However, if $p\gg n$ (i.e., the dimension is much ...
4
votes
1answer
223 views

Why is Linear Discriminant Analysis a linear classifier?

Question As the title suggests, I am wondering why we consider LDA a linear classifier. More specifically, I would like to know why LDA is considered a linear classifier in the following case: ...
5
votes
2answers
938 views

Why with two classes, LDA gives only one dimension?

I am working with dimensionality reduction algorithms. Linear Discriminant Analysis (LDA) is a supervised algorithm that takes into account the class label (which is not the case of PCA for example). ...
0
votes
0answers
15 views

Find the best “pair” of features to predict binary response with lda

Let's take the mtcars dataset: I want to find the 2 best features to predict vs 0 or ...
0
votes
0answers
52 views

How does SPSS “standardize” variables for discriminant analysis?

We are trying to reproduce the outcome of a discriminant analysis done in SPSS and run into a mystery. This link: https://stats.idre.ucla.edu/spss/output/discriminant-analysis/ explains that ...
1
vote
0answers
66 views

Linear Discriminant Analysis vs Quadratic Discriminant Analysis [duplicate]

I'm looking to find out peoples thoughts on LDA and QDA. When is it better to use one than the other? Also advantages and disadvantages of using both methods for classification purposes.
0
votes
1answer
25 views

Which analysis to use to discriminate morphometrics measurements from different species from 2 different environment?

So I have a dataset of measurements (lengths, surface areas, volumes...) from 3 species from 2 different environments, with 3 individuals per species. Can be summarised like that: ...
0
votes
0answers
47 views

LDA - direction which maximizes class separation

As far as I understood - at least form a very raw conceptual point of view, LDA (Linear Discriminant Analysis), when used as a dimensional reduction technique, does two things (I'll stick to the 2-...
1
vote
0answers
23 views

Non-Negative and Sparse LDA?

I am wondering if the concept of Non-Negative and Sparse Linear Discriminant Analysis exists? I recently found an algorithm for Non-Negative and Sparse Principal Component Analysis (on which the PC ...
0
votes
0answers
68 views

Why LDA objective function is non convex?

I have read an article. Now it is my question, Why below optimization problem is nonconvex(it is the LDA objective function)? And when we find the dual solution what does that mean for the above ...
0
votes
0answers
43 views

ٌWhy we use SVD in Penalized Discriminant Analysis?

I have read Penalized Discriminant Analysis article (Hastie and Tibshirani). Actually, I am new in this field and I have a couple of questions about the article. There are 3 criterions which were ...
3
votes
0answers
139 views

Rayleigh quotient, traces and LDA optimization problem

I've been working about Linear Discriminant Analysis the last weeks, and after reading many articles, I see some aspects of this problem not very clear. The LDA optimization problem is formulated by ...
3
votes
1answer
190 views

Autoencoders and Collaborative Filtering: Is one network per training sample really necessary?

I have been researching on how to apply neural networks to recommender systems and have come across this paper (AutoRec: Autoencoders Meet Collaborative Filtering by Sedhain et al.) where they model ...
4
votes
2answers
97 views

Derivation of $S_W^{-1} S_B$ during the calculation of LDA

I try to reason the computations during the search for the optimal weight vector $w$ during the calculations of LDA. Therefore I use several text books like: Kuhn, M. and Johnson, K. (2013) Applied ...
3
votes
0answers
84 views

Why are the discriminant axes in linear discriminant analysis (LDA) not orthogonal?

This may be a quite silly question and please correct me if I'm wrong. The discriminants (discriminant axes) are essentially eigenvectors of $\mathrm{Cov}_\mathrm{within}^{-1} \mathrm{Cov}_\mathrm{...
3
votes
1answer
127 views

Looking for good, recent examples of Discriminant Analysis (Linear, Quadratic or else) in applied research

We have looked at LDA/QDA several times during my stats masters coursework, but I'm not convinced that it's due to the usefulness of the techniques more than my school being stuck with a 20-year old ...
0
votes
1answer
243 views

Using chi-squared or binomial for triangle test of discrimination

Regarding beer taste testing. The wonderful people at Brulosophy do many experiments with beer, changing brew and fermentation methods and then using triangle tests to see whether there's a difference....
1
vote
1answer
83 views

Why does Fisher use covariance when only variance is needed?

With reference to the following image from here: (can not inline it due to unsupported format) https://wikimedia.org/api/rest_v1/media/math/render/svg/9af8aa035642689bb2004047416b069a15406447 If we ...