Discriminant (function) analysis (DA) is a dimensionality reduction and classification method. It finds low-dimensional subspace with the strongest class separation and uses it to perform classification. Most well-known is Linear Discriminant analysis (LDA) which provides linear borders of ...

learn more… | top users | synonyms

0
votes
0answers
9 views

Checking Multivariate Gaussian (normal) distribution before applying LDA

I have a data set with multiple predictors. I want to apply Linear Discriminant analysis (LDA) on my data. But before doing that I must confirm that my data is from multivariate Gaussian Distribution ...
0
votes
0answers
8 views

How are the between and within sum-of-squares matrices defined?

I was reading Tibishirani's paper, and on page 2 I came across the terms between and within sum-of-squares matrices. How are those matrices defined? Are they related to the Uncorrected Sums of Squares ...
0
votes
0answers
20 views

Fisher Discriminant Analysis vs ANOVA [closed]

Both FDA and ANOVA talks about minimizing within variance and maximizing across across variance. Lets say there are 3 classes for which feature f1 data is available. We can apply both of the above ...
0
votes
0answers
32 views

Common covariance matrix in linear discriminant analysis

Say I want to perform LDA classification involving three classes with within-class covariance matrices $$ \hat{\Sigma}_1 \,, \hat{\Sigma}_2 \, , \hat{\Sigma}_3$$ and that these matrices are calculated ...
0
votes
1answer
35 views

Qualitative implications of Linear Discriminant Analysis (LDA)

I'm a beginner to LDA, and my question is about its qualitative implications. Say I have two classes of medical data, already classified as: $C_1=\{x_{11},x_{12}, ...x_{1n_1}\}, C_2=\{x_{21}, x_{22} ...
0
votes
0answers
78 views

Discriminant function of 1 Nearest Neighbor

Consider the following question: We will consider the case of 1-nearest neighbor, and look at the details of computing the error probability. In this case, let us assume that we have two classes, ...
0
votes
1answer
25 views

Comparison of LDA vs KNN time complexity

Which algorithm has a better performance in terms of time complexity, LDA or KNN?
2
votes
0answers
27 views

Two-classes LDA on third class

I am trying to implement a $N$ classes classification with several 2-classes LDAs. I actually am using LDA as a projection method instead of classification, so it might be more a factor analysis. If ...
0
votes
0answers
10 views

How do I derive the Discriminant Function in Linear Discriminant Analysis

From An Introduction to Statistical Learning with Applications in R on page 143, the authors talk about obtaining the discriminant function in the case of LDA for >1 predictors. Assuming that we are ...
0
votes
0answers
9 views

Link between the FDA and LS-SVM

I am reading tutorial written by Johan Suykens:Least Squares Support Vector Machines On page 19,he mentions link with kernel Fisher Discriminant Analysis Project ...
0
votes
0answers
26 views

LDA and QDA main concepts

I've been studying LDA and QDA main concepts and already can implement those algorithms, but I'm not good at understanding the theory of this technique. Textbooks and Google gives me some properties ...
0
votes
0answers
41 views

Unsupervised clustering based on discriminant line

I have quite specific statistical problem, highly limited by its ecological interpretation. I have plenty of "time series" data - I need to link supression of photosythesis to the lack of light (PAR) ...
0
votes
0answers
8 views

how to define which of measured, non gaussian variables are effective in discriminate given groups?

I have eight groups (100 samples each) and 43 evaluated variables, with different distributions (some of them are right skewed, some left skewed, some with many zeros etc.). I'm trying to understand ...
0
votes
0answers
29 views

Difference between GMM classification and QDA

I know that every class has the same covariance matrix $\Sigma$ in linear discriminant analysis (LDA), and in quadratic discriminant analysis (QDA) they are different. When using gaussian mixture ...
5
votes
1answer
57 views

Interpretation of the cluster criterion $\operatorname{tr}(S_W^{-1}S_B)$

There is a cluster criterion defined as: $$\mathcal{C} = \operatorname{tr}(S_W^{-1}S_B) = \sum_{i=1}^d \lambda_i,$$ where $\operatorname{tr}$ is the trace, $S_W$ is the pooled within-group scatter ...
0
votes
1answer
60 views

LDA for dimensionality reduction usage

I have a original dataset with 70 samples, each sample with 96 features. The samples are labeled as 0 or 1. So I use linear discriminant analysis (LDA) to reduce the dimensionality of all the dataset, ...
1
vote
0answers
68 views

Using projected points from Linear Discriminant Analysis to generate probability density function

I am using Linear Discriminant Analysis technique to get the best possible separation between two distributions. I am using R for my programming. For LDA, I find the between-class scatter matrix(B) ...
2
votes
1answer
79 views

Comparing four formulations of class scatter matrices

I am trying to decide between / reconcile four formulations for class scatter matrices. The first from Duda et al. (2012), p.544 has (with symbols modified): $$m_i = \frac{1}{n_i} \sum_{x\in ...
0
votes
0answers
69 views

How to use LDA results for prediction? And how to assess model fit?

I am trying to understand how to interpret the results I get from LDA. Running from the iris dataset in R, I can see the discriminant coefficients are in the model and then I can plot the model to ...
2
votes
1answer
106 views

Meaning of “reconstruction error” in PCA and LDA

I am implementing PCA, LDA, and Naive Bayes, for compression and classification respectively (implementing both an LDA for compression and classification). I have the code written and everything ...
3
votes
2answers
124 views

How to build a decision tree with a constraint on sensitivity?

I am trying to develop a classification model on a sample of people which will discriminate between "Type A" and "Not-Type A" people. Due to external factors, the minimum sensitivity for this ...
2
votes
0answers
29 views

DFA in SPSS: Sorts effectively, but Box's M is still 0.000. Is the analysis worthless?

I am a geologist attempting to apply the discriminant function analysis to surface features I have mapped in ArcGIS. At the moment I have 4 dimensionless sorting variables calculated for each feature, ...
1
vote
1answer
126 views

Variable selection using cross-validated PLS model when permutation test shows lack of significance

I understand that the permutation test on PLS can help to detect overfitting of the PLS model. Usually if the p-value is greater than a criterion, say 0.05, it means that the model is overfitting and ...
0
votes
2answers
80 views

choosing between logistic and discriminant

I am looking at regularized logistic regression, (l1 and l2 at the moment) and regularized discriminant analysis. How do I compare the two? I was thinking of doing gcv on both methods over a set of ...
2
votes
0answers
67 views

Why does discriminant analysis prevent us from finding more than $K-1$ linear 'features' and what does it mean?

According to Bishop's Machine Learning and Pattern Recognition, the cost function for linear discriminant analysis (LDA) with $K>2$ classes is $$J(\mathbf w) = \mathrm{Tr}\left\{\left(\mathbf W ...
1
vote
0answers
41 views

Overfitting with diagonal matrix of covariances

Sample is a realisation of vector of random variables, each variable ~ $\mathcal N$. The number of samples is small, the length of sample is big. I would like to perform one-class LDA (linear ...
1
vote
0answers
41 views

Train & predict probabilities using LDA having multiple collinearities

I am trying to fit an LDA model and predict conditional probabilities of class membership with it. I believe I understand the basic method to do this using the covariance matrix and class means, but ...
5
votes
1answer
151 views

Seeming disagreement between learning sources about linear/quadratic and Fisher's discriminant analysis

I'm studying discriminant analysis, but I'm having a difficult time reconciling several different explanations. I believe I must be missing something, because I've never encountered this (seeming) ...
3
votes
0answers
39 views

What happens to linear discriminant analysis when $p>n$?

I have a general question regarding LDA (Fisher's linear discriminant analysis). What happens if the sample size $n$ is smaller than the dimensionality $p$ (number of predictors)? Is it possible to ...
0
votes
0answers
67 views

Linear Discriminant Analysis for dimensionality reduction - choosing the dimension

I'm using Linear Discriminant Analysis to do dimensionality reduction of a multi-class data. What is the best method to determine the "correct" number of dimensions? Can I use a method similar to PCA, ...
0
votes
0answers
25 views

How to see a Gaussian Discriminant Analysis (GDA) as a linear model for multiclass case?

In GDA we can assume that posterior probability for each of $K$ possible classes is Gaussian with same variance $\Sigma$, and different means $\mu_k$, ie. $$p(\mathbf x|C_k):\mathcal N(\mathbf ...
5
votes
2answers
376 views

Discriminant analysis vs logistic regression

I found some pros of discriminant analysis and I've got questions about them. So: When the classes are well-separated, the parameter estimates for logistic regression are surprisingly unstable. ...
1
vote
0answers
34 views

How generate data for classification tasks in R

At the last time i asked, is it possible in R to generate data on the given R -square, and as it turns out it's simple. ...
1
vote
1answer
60 views

How to calculate mahalanobis distance in SIMCA where different number of PCs are obtained for each class

I am working on a software that does SIMCA using mahalanobis distances with the following steps(excluding the classification of new objects for simplicity): Center each class individually Apply PCA ...
0
votes
0answers
31 views

Transformation to be used for continuous variable

I have a data set where I am doing a binary classification. I have close to 500 features and 200K observations. Now I also have few continuous variables as features. I don't think just using these ...
1
vote
1answer
124 views

Regularized discriminant analysis in Matlab

I am trying to do the 2-class classification using regularized discriminant analysis in Matlab using fitdiscr() function. The coefficients are stored in the object ...
0
votes
0answers
75 views

Cutoff value in linear discriminant analysis with two groups

I have a simple linear discrimininant analysis with two classes. Prior probabilitiest are fixed to 0.5 and number of cases is equal between groups. In this case the cutoff value could be calculated as ...
0
votes
0answers
49 views

Ensemble LDA on different feature spaces?

I'm working on a classification problem where I'd like to do the following: I have a space of features that live in $R^m$, and another set of features that are related that live in $R^n$. I want to ...
0
votes
1answer
94 views

Using PCA, clustering, and LDA together

After reading about both algorithms (Principal Component analysis and Linear Discriminant analysis), I started using them combined in a way which appeared intuitive to me. I have a data set that I ...
0
votes
0answers
18 views

1D score from 3-class LDA

I am using 3-class linear discriminant analysis on a data set. The 3 class labels correspond to a single value, with high, mid and low values (labels -1, 0, and 1). The 3-class LDA works much better ...
0
votes
1answer
129 views

What methodology does proc varclus use to reduce the number of variables

In statistics, we can use methods like principal component analysis, linear discriminant analysis for variable reduction. In SAS, there is a proc called VARCLUS which is used for variable reduction. ...
0
votes
0answers
55 views

LDA - Linear discriminant function

What is the formula used to compute the posterior probability for LDA in R ? I have a unbalanced class 97% to 3%. Does LDA is good in this case ?
2
votes
0answers
118 views

How to use LDA results for feature selection?

I am working on the Forest type mapping dataset which is available in the UCI machine learning repository. I have 27 features to predict the 4 types of forest. I am performing a Linear Discriminant ...
2
votes
1answer
71 views

What learning occurs in linear discriminant analysis?

From what I understand, linear discriminant analysis (LDA) has an objective function, where you try to find a matrix that maps data from a $p$-dimensional feature space to a $r$-dimensional feature ...
1
vote
0answers
23 views

Appropriateness of LDA for classification when majority of features are binary?

Am I correct in understanding that when using linear discriminant analysis, one assumes that the feature values are normally distributed? If so, does this mean LDA is very poorly suited for data ...
2
votes
0answers
71 views

Separation of points clouds via classification methods

I have multiple images from a 3D-Scanner in point cloud form. Part of the image is a fixture to hold the object to be scanned. I want to extract the object itself by classifying the fixture and the ...
0
votes
0answers
13 views

I want to identify what combination of discrete variables best classify into one of three groups

I have a series of images for patients who fall within three distinct diagnoses we have scored multiple features (about 8-10) for each patients into yes present or no not present how do i identify ...
1
vote
1answer
117 views

Conceptual undersanding of linear discriminant analysis

Can someone explain to a newbie the concepts of linear discriminant analysis? I am not looking for a technical implementation like this. I wish to understand it conceptually. I understand logistic ...
0
votes
1answer
104 views

Data normalization prior to PCA? [duplicate]

I want to get some intuition on normalization prior to feature selection with PCA. I'm sure z-normalization is a bad idea, since it normalizes the variances to 1 for each feature, PCA will be ...
1
vote
0answers
105 views

When is the scatter matrix in linear discriminant analysis singular?

In linear discriminant analysis (LDA), when there are fewer data instances than the number of dimensions (i.e., when the data matrix is of order $n \times m$ where $n$ is less than $m$), the ...