Discriminant (function) analysis (DA) is a dimensionality reduction and classification method. It finds low-dimensional subspace with the strongest class separation and uses it to perform classification. Most well-known is Linear Discriminant analysis (LDA) which provides linear borders of ...

learn more… | top users | synonyms

2
votes
0answers
22 views

Two-classes LDA on third class

I am trying to implement a $N$ classes classification with several 2-classes LDAs. I actually am using LDA as a projection method instead of classification, so it might be more a factor analysis. If ...
0
votes
0answers
9 views

How do I derive the Discriminant Function in Linear Discriminant Analysis

From An Introduction to Statistical Learning with Applications in R on page 143, the authors talk about obtaining the discriminant function in the case of LDA for >1 predictors. Assuming that we are ...
0
votes
0answers
9 views

Link between the FDA and LS-SVM

I am reading tutorial written by Johan Suykens:Least Squares Support Vector Machines On page 19,he mentions link with kernel Fisher Discriminant Analysis Project ...
0
votes
0answers
25 views

LDA and QDA main concepts

I've been studying LDA and QDA main concepts and already can implement those algorithms, but I'm not good at understanding the theory of this technique. Textbooks and Google gives me some properties ...
0
votes
0answers
41 views

Unsupervised clustering based on discriminant line

I have quite specific statistical problem, highly limited by its ecological interpretation. I have plenty of "time series" data - I need to link supression of photosythesis to the lack of light (PAR) ...
0
votes
0answers
7 views

how to define which of measured, non gaussian variables are effective in discriminate given groups?

I have eight groups (100 samples each) and 43 evaluated variables, with different distributions (some of them are right skewed, some left skewed, some with many zeros etc.). I'm trying to understand ...
0
votes
0answers
26 views

Difference between GMM classification and QDA

I know that every class has the same covariance matrix $\Sigma$ in linear discriminant analysis (LDA), and in quadratic discriminant analysis (QDA) they are different. When using gaussian mixture ...
5
votes
1answer
56 views

Interpretation of the cluster criterion $\operatorname{tr}(S_W^{-1}S_B)$

There is a cluster criterion defined as: $$\mathcal{C} = \operatorname{tr}(S_W^{-1}S_B) = \sum_{i=1}^d \lambda_i,$$ where $\operatorname{tr}$ is the trace, $S_W$ is the pooled within-group scatter ...
0
votes
1answer
56 views

LDA for dimensionality reduction usage

I have a original dataset with 70 samples, each sample with 96 features. The samples are labeled as 0 or 1. So I use linear discriminant analysis (LDA) to reduce the dimensionality of all the dataset, ...
1
vote
0answers
64 views

Using projected points from Linear Discriminant Analysis to generate probability density function

I am using Linear Discriminant Analysis technique to get the best possible separation between two distributions. I am using R for my programming. For LDA, I find the between-class scatter matrix(B) ...
2
votes
1answer
75 views

Comparing four formulations of class scatter matrices

I am trying to decide between / reconcile four formulations for class scatter matrices. The first from Duda et al. (2012), p.544 has (with symbols modified): $$m_i = \frac{1}{n_i} \sum_{x\in ...
0
votes
0answers
60 views

How to use LDA results for prediction? And how to assess model fit?

I am trying to understand how to interpret the results I get from LDA. Running from the iris dataset in R, I can see the discriminant coefficients are in the model and then I can plot the model to ...
2
votes
1answer
96 views

Meaning of “reconstruction error” in PCA and LDA

I am implementing PCA, LDA, and Naive Bayes, for compression and classification respectively (implementing both an LDA for compression and classification). I have the code written and everything ...
3
votes
2answers
111 views

How to build a decision tree with a constraint on sensitivity?

I am trying to develop a classification model on a sample of people which will discriminate between "Type A" and "Not-Type A" people. Due to external factors, the minimum sensitivity for this ...
2
votes
0answers
26 views

DFA in SPSS: Sorts effectively, but Box's M is still 0.000. Is the analysis worthless?

I am a geologist attempting to apply the discriminant function analysis to surface features I have mapped in ArcGIS. At the moment I have 4 dimensionless sorting variables calculated for each feature, ...
1
vote
1answer
119 views

Variable selection using cross-validated PLS model when permutation test shows lack of significance

I understand that the permutation test on PLS can help to detect overfitting of the PLS model. Usually if the p-value is greater than a criterion, say 0.05, it means that the model is overfitting and ...
0
votes
2answers
79 views

choosing between logistic and discriminant

I am looking at regularized logistic regression, (l1 and l2 at the moment) and regularized discriminant analysis. How do I compare the two? I was thinking of doing gcv on both methods over a set of ...
2
votes
0answers
67 views

Why does discriminant analysis prevent us from finding more than $K-1$ linear 'features' and what does it mean?

According to Bishop's Machine Learning and Pattern Recognition, the cost function for linear discriminant analysis (LDA) with $K>2$ classes is $$J(\mathbf w) = \mathrm{Tr}\left\{\left(\mathbf W ...
1
vote
0answers
40 views

Overfitting with diagonal matrix of covariances

Sample is a realisation of vector of random variables, each variable ~ $\mathcal N$. The number of samples is small, the length of sample is big. I would like to perform one-class LDA (linear ...
1
vote
0answers
41 views

Train & predict probabilities using LDA having multiple collinearities

I am trying to fit an LDA model and predict conditional probabilities of class membership with it. I believe I understand the basic method to do this using the covariance matrix and class means, but ...
5
votes
1answer
137 views

Seeming disagreement between learning sources about linear/quadratic and Fisher's discriminant analysis

I'm studying discriminant analysis, but I'm having a difficult time reconciling several different explanations. I believe I must be missing something, because I've never encountered this (seeming) ...
3
votes
0answers
36 views

What happens to linear discriminant analysis when $p>n$?

I have a general question regarding LDA (Fisher's linear discriminant analysis). What happens if the sample size $n$ is smaller than the dimensionality $p$ (number of predictors)? Is it possible to ...
0
votes
0answers
58 views

Linear Discriminant Analysis for dimensionality reduction - choosing the dimension

I'm using Linear Discriminant Analysis to do dimensionality reduction of a multi-class data. What is the best method to determine the "correct" number of dimensions? Can I use a method similar to PCA, ...
0
votes
0answers
24 views

How to see a Gaussian Discriminant Analysis (GDA) as a linear model for multiclass case?

In GDA we can assume that posterior probability for each of $K$ possible classes is Gaussian with same variance $\Sigma$, and different means $\mu_k$, ie. $$p(\mathbf x|C_k):\mathcal N(\mathbf ...
5
votes
2answers
313 views

Discriminant analysis vs logistic regression

I found some pros of discriminant analysis and I've got questions about them. So: When the classes are well-separated, the parameter estimates for logistic regression are surprisingly unstable. ...
1
vote
0answers
34 views

How generate data for classification tasks in R

At the last time i asked, is it possible in R to generate data on the given R -square, and as it turns out it's simple. ...
1
vote
0answers
54 views

How to calculate mahalanobis distance in SIMCA where different number of PCs are obtained for each class

I am working on a software that does SIMCA using mahalanobis distances with the following steps(excluding the classification of new objects for simplicity): Center each class individually Apply PCA ...
0
votes
0answers
29 views

Transformation to be used for continuous variable

I have a data set where I am doing a binary classification. I have close to 500 features and 200K observations. Now I also have few continuous variables as features. I don't think just using these ...
1
vote
1answer
117 views

Regularized discriminant analysis in Matlab

I am trying to do the 2-class classification using regularized discriminant analysis in Matlab using fitdiscr() function. The coefficients are stored in the object ...
0
votes
0answers
68 views

Cutoff value in linear discriminant analysis with two groups

I have a simple linear discrimininant analysis with two classes. Prior probabilitiest are fixed to 0.5 and number of cases is equal between groups. In this case the cutoff value could be calculated as ...
0
votes
0answers
47 views

Ensemble LDA on different feature spaces?

I'm working on a classification problem where I'd like to do the following: I have a space of features that live in $R^m$, and another set of features that are related that live in $R^n$. I want to ...
0
votes
1answer
88 views

Using PCA, clustering, and LDA together

After reading about both algorithms (Principal Component analysis and Linear Discriminant analysis), I started using them combined in a way which appeared intuitive to me. I have a data set that I ...
0
votes
0answers
18 views

1D score from 3-class LDA

I am using 3-class linear discriminant analysis on a data set. The 3 class labels correspond to a single value, with high, mid and low values (labels -1, 0, and 1). The 3-class LDA works much better ...
0
votes
1answer
104 views

What methodology does proc varclus use to reduce the number of variables

In statistics, we can use methods like principal component analysis, linear discriminant analysis for variable reduction. In SAS, there is a proc called VARCLUS which is used for variable reduction. ...
0
votes
0answers
55 views

LDA - Linear discriminant function

What is the formula used to compute the posterior probability for LDA in R ? I have a unbalanced class 97% to 3%. Does LDA is good in this case ?
2
votes
0answers
110 views

How to use LDA results for feature selection?

I am working on the Forest type mapping dataset which is available in the UCI machine learning repository. I have 27 features to predict the 4 types of forest. I am performing a Linear Discriminant ...
2
votes
1answer
63 views

What learning occurs in linear discriminant analysis?

From what I understand, linear discriminant analysis (LDA) has an objective function, where you try to find a matrix that maps data from a $p$-dimensional feature space to a $r$-dimensional feature ...
1
vote
0answers
21 views

Appropriateness of LDA for classification when majority of features are binary?

Am I correct in understanding that when using linear discriminant analysis, one assumes that the feature values are normally distributed? If so, does this mean LDA is very poorly suited for data ...
2
votes
0answers
66 views

Separation of points clouds via classification methods

I have multiple images from a 3D-Scanner in point cloud form. Part of the image is a fixture to hold the object to be scanned. I want to extract the object itself by classifying the fixture and the ...
0
votes
0answers
13 views

I want to identify what combination of discrete variables best classify into one of three groups

I have a series of images for patients who fall within three distinct diagnoses we have scored multiple features (about 8-10) for each patients into yes present or no not present how do i identify ...
1
vote
1answer
115 views

Conceptual undersanding of linear discriminant analysis

Can someone explain to a newbie the concepts of linear discriminant analysis? I am not looking for a technical implementation like this. I wish to understand it conceptually. I understand logistic ...
0
votes
1answer
102 views

Data normalization prior to PCA? [duplicate]

I want to get some intuition on normalization prior to feature selection with PCA. I'm sure z-normalization is a bad idea, since it normalizes the variances to 1 for each feature, PCA will be ...
1
vote
0answers
94 views

When is the scatter matrix in linear discriminant analysis singular?

In linear discriminant analysis (LDA), when there are fewer data instances than the number of dimensions (i.e., when the data matrix is of order $n \times m$ where $n$ is less than $m$), the ...
3
votes
1answer
301 views

How LDA, a classification technique, also serves as dimensionality reduction technique like PCA

In this article , the author links linear discriminant analysis (LDA) to principal component analysis (PCA). With my limited knowledge, I am not able to follow how LDA can be somewhat similar to PCA. ...
0
votes
0answers
17 views

Structural partition for cross validation

Can the partitioning of data for cross validation be used structurally to assess the transfer ability of an effect of one factor across another factor? A MANOVA performed on multivariate data reveals ...
0
votes
1answer
109 views

Fisher Projection vs Linear Discriminant Analysis [closed]

Basically, I am confused between Fisher and LDA. Looking for differences between the two. How is the Fischer projection computed in R?
1
vote
1answer
274 views

Difference between within-group and between-group covariance matrices in linear discriminant analysis

Could someone explain to me the difference between within-group covariance matrix and between-group covariance matrix in the context of linear discriminant analysis?
0
votes
0answers
28 views

Sphering and Centering Data

If I have a bunch of data points that like in p-dimensions, what does it mean if the data is sphered and centered. I never really understood this concept and was wondering if anyone could explain it ...
2
votes
1answer
216 views

Why are discriminant analysis results in R (lda) and SPSS different?: Constant term

I tried discriminant analysis with lda() in R and in SPSS, but the scalings were different, why? N, how to get (Constant) with R like SPSS result? data: ...
2
votes
2answers
542 views

Best way to bin continuous data

I have a data frame with 1 vector of integers and 1 as a character factor like so: I have created a linear model that shows a relationship between age and party affiliation. I now want to determine ...