Skip to main content

Questions tagged [discriminant-analysis]

Linear Discriminant Analysis (LDA) is a dimensionality reduction and classification method. It finds low-dimensional subspace with the strongest class separation and uses it to perform classification. Use this tag for quadratic DA (QDA) too.

Filter by
Sorted by
Tagged with
0 votes
0 answers
15 views

Meaning of within-class Covariance in Linear Discriminant Analysis Dimensionality Reduction

In section 4.3.3 of Elements of Statistical Learning by Hastie, Tibshirani, and Friedman the authors listed a procedure to reduce the dimensions of an input matrix $\mathbf{X}$, first using Linear ...
Jack Guan's user avatar
  • 103
4 votes
5 answers
158 views

Is it valid to exhaustively test all possible combinations of features to find the best combination?

I have about 1000 labelled observations from about 50 subjects responding physiologically under different situations and am trying to classify the situation (usually into three classes of roughly ...
user1596274's user avatar
0 votes
0 answers
23 views

Creating a discriminant function from one known group and one mixed group

I have a dataset composed of captures of individuals (ringed birds), all from the same species. Because the captures took place during spring, summer and autumn, we have trapped breeding and migrating ...
ElsouJ's user avatar
  • 1
1 vote
1 answer
30 views

Can AdaBoost discriminate the data?

Assume that you have an image $X$ and you want to search a specific small object inside the image $X$. It's only one object. So you training an AdaBoost model with only one valid train sample and the ...
euraad's user avatar
  • 425
3 votes
1 answer
96 views

Two-class LDA with multiple variables: can we separate classes and know the weight of variables?

So far I have performed an LDA having two classes, but I'm really struggling with certain aspectos of the analysis. I am working with biomedical data in which I would like to classify in k=2 groups: ...
Javier Hernando's user avatar
0 votes
0 answers
78 views

Is my understanding of dimension reduction in Linear Discriminant Analysis correct?

Here is my understanding of how dimension reduction in LDA works: We have $n$ samples each with $p$ features assigned to $k$ classes. We use the sample mean $\mu_j$ of each class and the pooled ...
Steven Gubkin's user avatar
1 vote
0 answers
143 views

How to balance PCA and LDA in subspace learning?

PCA is a generative model, by which input images or data can be reconstructed. LDA (Linear Discriminant Analysis) is a discriminative model, which extracts better features for classification. How to ...
chickensoup's user avatar
1 vote
0 answers
34 views

Linear Discriminant Analysis with unlabeled data

In section 4.4.5 "Logistic regression or LDA?" of Elements of Statistical Learning by Friedman, Tibshirani and Hastie, it is claimed the following: From the mixture formulation [that is, ...
Sergio's user avatar
  • 336
0 votes
0 answers
195 views

A lot of variables in a Quadratic Discriminant Analysis

I'm trying to make a Quadratic Discriminant Analysis in R, but appears the follow mistake: "some group is too small for 'qda'". I was reading about it and I concluded that I have more ...
César Rendón Mayorga's user avatar
0 votes
0 answers
12 views

Does the ID class vector change in Kernel Linear Discriminant Analysis?

Assume that I have a matrix $X$ that has the size $m * n$ and a class ID vector with the length $n$. If I want to apply Kernel Linear Discriminant Analysis (KLDA) onto the matrix $X$ and vector $y$, ...
euraad's user avatar
  • 425
0 votes
0 answers
26 views

How do I associate or assign a large amount of continuous variables with zero-heavy distributions to different groups?

I have a dataset with about 70 continuous numeric variables. I have about 80 samples which divide more or less equally into two groups. I want to figure out which of these variables is most strongly ...
Tal's user avatar
  • 11
1 vote
0 answers
51 views

Is Fisher's discriminant analysis equivalent to the Bayes optimal LDA when the no. of classes is greater than two and covariances are all equal?

P.S. While I gave a brief background to make the question complete, informed readers can move to the questions 1 and 2 towards the end of this post, right after 'what are not clear to me are:'. Fisher'...
Mathmath's user avatar
  • 761
1 vote
0 answers
82 views

Importance of linear discriminants to classification at a given point

Many outstanding answers here detail the fundamentals of linear discriminant analysis. These include descriptions of its use in dimensionality reduction, an explanation of classification using Bayes' ...
athankfulguest's user avatar
0 votes
0 answers
36 views

LDA: sample mean of two distributions are proportional

I came up with a discussion around what does it mean that the sample mean of two distributions are proportional. This discussion started when talking about linear discriminant analysis and how this ...
GGChe's user avatar
  • 165
2 votes
1 answer
92 views

So how can I project the data with the eigenvectors from LDA?

I have data that look like this. And my goal is to reduce this 3D dimension into 2D dimension so it might looks like this. Turning the angle so the distance between all classes becomes maximum. So ...
euraad's user avatar
  • 425
0 votes
0 answers
43 views

Best subset of variables to perform discriminant analysis

I have around 33 variables, and 300 observations, although of some variables there are some missing data. I would like to obtain the best subset of variables which can separate the best 3 categories ...
Adrián Valls Carbó's user avatar
4 votes
1 answer
178 views

Why is the Scaling Matrix in LDA unnormalized?

I was carrying out LDA (linear Discriminant Analysis) and noticed that the Scaling matrix produced by R is not normalized. Here is an example: ...
Onyambu's user avatar
  • 516
1 vote
0 answers
109 views

Could I use LDA (Linear discriminant analysis) for outlier detection?

Could I use LDA method to separate outlier from major points? I want to find outlier from certain data with LDA, but I couldn't find use of LDA for outlier detection. Basically I want do the work like ...
Jerison K's user avatar
1 vote
0 answers
70 views

Reduced Rank Linear Discriminant Analysis vs Fisher Discriminant as mentioned in Element of Statistical learning section 4.3.3

In ESL section 4.3.3 , Author gives three steps for finding optimal subspaces using LDA as below compute the K × p matrix of class centroids M and the common covariance matrix W (for within-class ...
user9451426's user avatar
2 votes
2 answers
141 views

Is this description of linear discriminant analysis (LDA) correct?

Hang Nguyen writes the following in "Machine learning basics (part 14): Linear Discriminant Analysis": If there are two classes then the LDA draws one hyperplane and projects the data onto ...
Richard Hardy's user avatar
1 vote
0 answers
187 views

Shipley's d separation test in piecewise structural equation model

I have created a piecewise structural equation model for when the temperature exceeds 0 degrees Celsius, and how that relates to the green up day across Northern Europe. I have standardised all data <...
Thomas's user avatar
  • 528
1 vote
1 answer
401 views

Is linear discriminant analysis a supervised classifier or dimensionality reduction?

On page 147 of ISLR 2nd Edition, the author is talking about LDA and comparing it to a Bayes Classifier. This leads me to believe LDA is a machine learning algorithm for supervised classification. ...
Katsu's user avatar
  • 1,011
1 vote
1 answer
19 views

How do I justify this QDA expression for $a_k, ~b_{kj} $and $c_{jkl}$?

I need to solve the equation of $$\log\left(\frac{\operatorname{Pr}(Y=k|X=x)}{\operatorname{Pr}(Y=K|X=x)}\right)$$ $$=\log\left(\frac{\pi_k \exp\left((x-\mu_k)^T|\Sigma|^{-1}(x-\mu_k)\right)} {\pi_K \...
therickster's user avatar
1 vote
0 answers
41 views

linear transformation to simplify linear discriminant analysis

In a lecture on Linear Discriminant Analysis, it was mentioned that data given as $$ x|y=0 \sim N(\mu_0,\Sigma)\\ x|y=1 \sim N(\mu_1,\Sigma) $$ can be trnsformed by means of a linear transformation $\...
kara890's user avatar
  • 159
1 vote
0 answers
372 views

Show Quadratic Discriminant Analysis (QDA) implies a logistic regression model

I am attempting to show that QDA implies a logistic regression model of the form $\log(\frac{P(Y=1 \mid X = x)}{1-P(Y=1 \mid X = x)}) = \beta_0 + \beta_1x + \beta_2x^2$, where there is only one ...
feonyte's user avatar
  • 69
2 votes
1 answer
81 views

Linear discriminant analysis-Do we maximize posterior or joint distribution?

My understanding is that in LDA, we maximize the joint density $ P(X,y) $ using $P(X|y)$ rather than posterior density $P(y|X)$ using $P(X|y)$. In the book "Introduction to Statistical learning ...
Madhuresh's user avatar
0 votes
0 answers
47 views

How to find a decomposition of multivariate X along which y varies the most?

I'm looking for an existing algorithm which carries out the task shown in the title. My use-case in other words: I have a set of continuous independent variables (X) and a continuous dependent ...
Haliaetus's user avatar
  • 123
2 votes
1 answer
331 views

Factor analysis and Dimensional Reduction difference

In exploratory factor analysis, we can clearly see the weightage of the factors on different features. Following this, we can understand the extent to which factors have an influence on the features. ...
Marble's user avatar
  • 121
0 votes
1 answer
160 views

Discriminant validity

My question is about the need for this validation. I have highly correlated constructs in a cfa model fitted using the lavaan library in R, so the correlation is greater than the root of the AVE. From ...
Marcelo Rodrigues's user avatar
1 vote
1 answer
141 views

How can we use shannon entropy to discriminate between two similar probability distribution function?

I studied two papers related to discriminating between two similar distributions using Shannon entropy. But both of them had different views. Can anyone explain what would be the basic flow of idea to ...
nishant's user avatar
  • 11
0 votes
0 answers
143 views

My data can be approximated with normal distribution (multimodal). How can I find the reasons and explain this behaviour?

I use DeLonge method to compare two ROC AUCS. The result of it is Z-score. Both ROC AUCs obtained from LDA (linear discriminant analysis) from sklearn package. The ...
Arzental's user avatar
0 votes
1 answer
120 views

What is the difference between Pi and Pi(x) in the Bayesian Classification context?

What is the difference between Pi and Pi(x) in the Bayesian Classification context? Is it true to say that Pi is the marginal or prior probability? Is it true to say that Pi(x) is the posterior ...
Python_Guy's user avatar
2 votes
0 answers
2k views

How to interpret the output of LDA (discriminant analysis) in R

I'm doing a study where I have 4 groups of people who take 5 different psychological tests. I'm interested in determining what test (or tests) is better at determining in which group a person is in. ...
aprendiz's user avatar
  • 123
1 vote
0 answers
162 views

Does linear discriminant analysis expands or only rotate?

Assume that we have two classes, $X$ and $Y$ and we find the mean $X_\mu$ and $Y_\mu$ and variance $X_\sigma$ and $Y_\sigma$. With that, we could use linear discriminant analysis to expend the ...
euraad's user avatar
  • 425
0 votes
1 answer
378 views

Finding Correlations between 2 Multivariate data sets

I have two data sets, and both are MultiVariate datasets The first dataset has a format as below, with the first column being the country of origin (only two countries, so binary classifier) of a ...
WOT's user avatar
  • 3
2 votes
0 answers
517 views

Bias-variance trade-off between LDA and QDA w.r.t. dimensionality

Consider the bias-variance trade-off between linear discriminant analysis (LDA) and quadratic discriminant analysis (QDA). Switching from QDA to LDA will generally yield a reduction in variance. The ...
Richard Hardy's user avatar
0 votes
1 answer
46 views

Repeated LDA for finding multiple discriminant dimensions

I am not very familiar with LDA but was interested in using it to find multiple, orthogonal discriminant dimensions for data with 2 class labels. I understand that standard that the point of LDA is to ...
user3235916's user avatar
3 votes
0 answers
64 views

Fisherfaces/Linear Discriminat Analysis - What are those faces supposed to be?

I see a lot of weird blue/gray/green faces when I search for "firsherfaces" at Google. I see faces like this and my question is simple: What are those faces supposed to be? Are they some ...
euraad's user avatar
  • 425
2 votes
1 answer
35 views

Discriminant Analysis for Non-Gaussian non-same distribution of features in classes

I am trying to understand discriminant analysis. say i have 2 classes and f1(x) and f2(x) are non-gaussian and do not have the same distribution as well. How do i get the discriminant function?
Srinag Vinil Tummala's user avatar
10 votes
2 answers
3k views

Intuition for why LDA is a special case of naive Bayes

The naive Bayes classifier assumes the regressors to be mutually independent, while linear discriminant analysis (LDA) allows them to be correlated. James et al. "An Introduction to Statistical ...
Richard Hardy's user avatar
0 votes
0 answers
1k views

LDA Feature Importantance Extraction - sklearn Python

I have run LDA from sklearn on a dataset and I have gotten pretty good separation between classes. Typically I can extract feature importance to determine which contributing factor is most important. ...
user334505's user avatar
1 vote
0 answers
39 views

Difference between discriminant analysis and neural network (one layer)

So I am doing some research on this topic and I "hit a wall" with this question. I managed to find some papers on using DA vs NN on data but i didn't find anything math related. Maybe ...
Roman's user avatar
  • 11
2 votes
0 answers
35 views

LDA followed by multiple PLS models OR PLS2 to include both the continuous dependent variable and the categorical variable

I have 1200 spectral Xvariables. I use PLS-LDA to reduce Xvariables and classify them in groups (contaminants). After that I need to quantify the "amount" of contaminant by PLS. First I ...
Piet Godaard's user avatar
1 vote
0 answers
104 views

Is the logistic regression with quadratic and interractions terms (special case) similar to QDA? And what about the prediction performance?

Beyond the fact that the the two methods have different assumptions : Logistic on the residuals extrem value distribution & utility theory. QDA on the predictors multivariate gaussian ...
Bentorcha Karim's user avatar
0 votes
1 answer
656 views

Latent profile analysis and Factor Analysis

How different is Latent Profile Analysis from Factor Analysis and what is the role of Multinomial Regression ? Does anybody have any intuitive ideas to understand the difference? Thanks.
Science11's user avatar
  • 577
3 votes
1 answer
411 views

Linear Discriminant Analysis- Derivation of Discriminant Function

Given that, we want to maximize the posterior probability, for the expression (1) for k, I wan't to know how the expression (2) is obtained: My understanding (may be wrong) is that the expansion ...
Prateek Pandey's user avatar
1 vote
0 answers
870 views

Why absolute value of eigenvalues are used in PCA or LDA?

In PCA and LDA techniques, eigenvectors with the $k$ largest eigenvalues give principal components. However, when selecting these eigenvalues, are they to be sorted by the absolute value (regardless ...
Ananda's user avatar
  • 11
5 votes
1 answer
103 views

Least Squares Solution involving regularizer and weighted sum

I have come across the following cost function: $$ \text{min}_a\ \ (a^Tx^{(1)} -1)^2 + \sum_{j=1}^M \alpha_j (a^Tx_j^{(2)} +1)^2 + \frac{\lambda}{2}||a||^2 $$ This is a minimization over weight vector ...
DarkLink's user avatar
  • 217
3 votes
0 answers
187 views

For a generative model, how is modelling p(X,Y) equivalent to modelling P(X|Y=y)?

On the Wikipedia page for generative models it gives the following definitions of a generative model: (X is an observable variable, Y is the target variable) 1) A generative model is a model of the ...
apprentice9's user avatar
5 votes
1 answer
410 views

Is classification using linear regression called logistic regression or linear disriminant analysis?

I have heard people describe logistic regression as linear regression except as it is deployed for classification. But I have heard the exact same comment about LDA (linear discriminant analysis). Out ...
John S's user avatar
  • 135

1
2 3 4 5
10