Discriminant (function) analysis (DA) is a dimensionality reduction and classification method. It finds low-dimensional subspace with the strongest class separation and uses it to perform classification. Most well-known is Linear Discriminant analysis (LDA) which provides linear borders of ...

learn more… | top users | synonyms

0
votes
0answers
9 views

Replying the canonical discriminant analysis on the test set [on hold]

I run a canonical discriminant analysis following those 2 video lessons on Coursera. Now, I would like to test the accuracy of the clustering methodology on a test set by replicating the canonical ...
1
vote
0answers
16 views

Compute a Quadratic discriminant analysis (QDA) in R assuming not normal data and missing information

In this course, the professor is saying that we can compute a QDA with missing data points and non-normal data (even if this assumption can be violated). But the problem is that I don't know any ...
0
votes
1answer
31 views

Confusion Matrix values of zero

My question is almost exaclty the same as this one: Suspicious Amount of Zeros in Confusion Matrix, but no one seems to have responded so I figure I'll ask again. I've got a LOO confusion matrix that ...
0
votes
0answers
33 views

Using factor scores in discriminant analysis

I'm looking at indicators of earnings fraud in publicly-held corporations. My classification is dichotomous, Yes, company is likely to engage in EF, or no, it's not. I have 15 financial variables ...
-1
votes
0answers
6 views

Chi-Square Difference test, set correlation to one [migrated]

I'm working on my master thesis and I'm not sure how to interpret an error message by R. I have 5 latent Variables (DI, PR, BQ, RA and RV). Since the Fornell Larcker Kriterium is in some cases not met,...
0
votes
0answers
39 views
1
vote
0answers
33 views

Are PLS-DA and PLS-LDA the same?

Seems like a trivial question but one for which I can't seem to find an answer. Are PLS-DA (partial least squares discriminant analysis) and PLS-LDA (partial least squares followed by linear ...
2
votes
0answers
49 views

Unstable projection in LDA space in $n<p$ situation [duplicate]

I'm trying to classify (LDA) few samples (n=12) in a high dimensional feature space (p=24) into 3 classes. First I reduced the ...
0
votes
0answers
21 views

Does Gaussian discriminant analysis and linear discriminant analysis refer the same algorithm?

I'm pretty new to LDA and I came across other terminology called Gaussian discriminant analysis elsewhere. Since LDA assumes the normality or normal distribution of the data which is same as ...
0
votes
0answers
14 views

Checking Multivariate Gaussian (normal) distribution before applying LDA

I have a data set with multiple predictors. I want to apply Linear Discriminant analysis (LDA) on my data. But before doing that I must confirm that my data is from multivariate Gaussian Distribution ....
0
votes
0answers
8 views

How are the between and within sum-of-squares matrices defined?

I was reading Tibishirani's paper, and on page 2 I came across the terms between and within sum-of-squares matrices. How are those matrices defined? Are they related to the Uncorrected Sums of Squares ...
0
votes
0answers
33 views

Common covariance matrix in linear discriminant analysis

Say I want to perform LDA classification involving three classes with within-class covariance matrices $$ \hat{\Sigma}_1 \,, \hat{\Sigma}_2 \, , \hat{\Sigma}_3$$ and that these matrices are calculated ...
1
vote
1answer
44 views

Qualitative implications of Linear Discriminant Analysis (LDA)

I'm a beginner to LDA, and my question is about its qualitative implications. Say I have two classes of medical data, already classified as: $C_1=\{x_{11},x_{12}, ...x_{1n_1}\}, C_2=\{x_{21}, x_{22} ....
0
votes
0answers
78 views

Discriminant function of 1 Nearest Neighbor

Consider the following question: We will consider the case of 1-nearest neighbor, and look at the details of computing the error probability. In this case, let us assume that we have two classes, $\...
0
votes
1answer
45 views

Comparison of LDA vs KNN time complexity

Which algorithm has a better performance in terms of time complexity, LDA or KNN?
2
votes
0answers
35 views

Two-classes LDA on third class

I am trying to implement a $N$ classes classification with several 2-classes LDAs. I actually am using LDA as a projection method instead of classification, so it might be more a factor analysis. If ...
0
votes
0answers
15 views

How do I derive the Discriminant Function in Linear Discriminant Analysis

From An Introduction to Statistical Learning with Applications in R on page 143, the authors talk about obtaining the discriminant function in the case of LDA for >1 predictors. Assuming that we are ...
0
votes
0answers
10 views

Link between the FDA and LS-SVM

I am reading tutorial written by Johan Suykens:Least Squares Support Vector Machines On page 19,he mentions link with kernel Fisher Discriminant Analysis Project ...
0
votes
0answers
28 views

LDA and QDA main concepts

I've been studying LDA and QDA main concepts and already can implement those algorithms, but I'm not good at understanding the theory of this technique. Textbooks and Google gives me some properties ...
0
votes
0answers
41 views

Unsupervised clustering based on discriminant line

I have quite specific statistical problem, highly limited by its ecological interpretation. I have plenty of "time series" data - I need to link supression of photosythesis to the lack of light (PAR) ...
0
votes
0answers
8 views

how to define which of measured, non gaussian variables are effective in discriminate given groups?

I have eight groups (100 samples each) and 43 evaluated variables, with different distributions (some of them are right skewed, some left skewed, some with many zeros etc.). I'm trying to understand ...
0
votes
0answers
37 views

Difference between GMM classification and QDA

I know that every class has the same covariance matrix $\Sigma$ in linear discriminant analysis (LDA), and in quadratic discriminant analysis (QDA) they are different. When using gaussian mixture ...
5
votes
1answer
58 views

Interpretation of the cluster criterion $\operatorname{tr}(S_W^{-1}S_B)$

There is a cluster criterion defined as: $$\mathcal{C} = \operatorname{tr}(S_W^{-1}S_B) = \sum_{i=1}^d \lambda_i,$$ where $\operatorname{tr}$ is the trace, $S_W$ is the pooled within-group scatter ...
0
votes
1answer
74 views

LDA for dimensionality reduction usage

I have a original dataset with 70 samples, each sample with 96 features. The samples are labeled as 0 or 1. So I use linear discriminant analysis (LDA) to reduce the dimensionality of all the dataset, ...
1
vote
0answers
72 views

Using projected points from Linear Discriminant Analysis to generate probability density function

I am using Linear Discriminant Analysis technique to get the best possible separation between two distributions. I am using R for my programming. For LDA, I find the between-class scatter matrix(B) ...
2
votes
1answer
82 views

Comparing four formulations of class scatter matrices

I am trying to decide between / reconcile four formulations for class scatter matrices. The first from Duda et al. (2012), p.544 has (with symbols modified): $$m_i = \frac{1}{n_i} \sum_{x\in \...
0
votes
0answers
83 views

How to use LDA results for prediction? And how to assess model fit?

I am trying to understand how to interpret the results I get from LDA. Running from the iris dataset in R, I can see the discriminant coefficients are in the model and then I can plot the model to ...
2
votes
1answer
139 views

Meaning of “reconstruction error” in PCA and LDA

I am implementing PCA, LDA, and Naive Bayes, for compression and classification respectively (implementing both an LDA for compression and classification). I have the code written and everything ...
3
votes
2answers
142 views

How to build a decision tree with a constraint on sensitivity?

I am trying to develop a classification model on a sample of people which will discriminate between "Type A" and "Not-Type A" people. Due to external factors, the minimum sensitivity for this ...
2
votes
0answers
32 views

DFA in SPSS: Sorts effectively, but Box's M is still 0.000. Is the analysis worthless?

I am a geologist attempting to apply the discriminant function analysis to surface features I have mapped in ArcGIS. At the moment I have 4 dimensionless sorting variables calculated for each feature, ...
1
vote
1answer
152 views

Variable selection using cross-validated PLS model when permutation test shows lack of significance

I understand that the permutation test on PLS can help to detect overfitting of the PLS model. Usually if the p-value is greater than a criterion, say 0.05, it means that the model is overfitting and ...
0
votes
2answers
83 views

choosing between logistic and discriminant

I am looking at regularized logistic regression, (l1 and l2 at the moment) and regularized discriminant analysis. How do I compare the two? I was thinking of doing gcv on both methods over a set of ...
2
votes
0answers
68 views

Why does discriminant analysis prevent us from finding more than $K-1$ linear 'features' and what does it mean?

According to Bishop's Machine Learning and Pattern Recognition, the cost function for linear discriminant analysis (LDA) with $K>2$ classes is $$J(\mathbf w) = \mathrm{Tr}\left\{\left(\mathbf W \...
1
vote
0answers
42 views

Overfitting with diagonal matrix of covariances

Sample is a realisation of vector of random variables, each variable ~ $\mathcal N$. The number of samples is small, the length of sample is big. I would like to perform one-class LDA (linear ...
1
vote
0answers
41 views

Train & predict probabilities using LDA having multiple collinearities

I am trying to fit an LDA model and predict conditional probabilities of class membership with it. I believe I understand the basic method to do this using the covariance matrix and class means, but ...
5
votes
1answer
173 views

Seeming disagreement between learning sources about linear/quadratic and Fisher's discriminant analysis

I'm studying discriminant analysis, but I'm having a difficult time reconciling several different explanations. I believe I must be missing something, because I've never encountered this (seeming) ...
3
votes
0answers
42 views

What happens to linear discriminant analysis when $p>n$?

I have a general question regarding LDA (Fisher's linear discriminant analysis). What happens if the sample size $n$ is smaller than the dimensionality $p$ (number of predictors)? Is it possible to ...
0
votes
0answers
75 views

Linear Discriminant Analysis for dimensionality reduction - choosing the dimension

I'm using Linear Discriminant Analysis to do dimensionality reduction of a multi-class data. What is the best method to determine the "correct" number of dimensions? Can I use a method similar to PCA, ...
0
votes
0answers
26 views

How to see a Gaussian Discriminant Analysis (GDA) as a linear model for multiclass case?

In GDA we can assume that posterior probability for each of $K$ possible classes is Gaussian with same variance $\Sigma$, and different means $\mu_k$, ie. $$p(\mathbf x|C_k):\mathcal N(\mathbf \mu_k,\...
5
votes
2answers
415 views

Discriminant analysis vs logistic regression

I found some pros of discriminant analysis and I've got questions about them. So: When the classes are well-separated, the parameter estimates for logistic regression are surprisingly unstable. ...
1
vote
0answers
37 views

How generate data for classification tasks in R

At the last time i asked, is it possible in R to generate data on the given R -square, and as it turns out it's simple. http://stackoverflow.com/questions/33920813/how-to-generate-data-on-the-a-given-...
1
vote
1answer
66 views

How to calculate mahalanobis distance in SIMCA where different number of PCs are obtained for each class

I am working on a software that does SIMCA using mahalanobis distances with the following steps(excluding the classification of new objects for simplicity): Center each class individually Apply PCA ...
0
votes
0answers
34 views

Transformation to be used for continuous variable

I have a data set where I am doing a binary classification. I have close to 500 features and 200K observations. Now I also have few continuous variables as features. I don't think just using these ...
1
vote
1answer
131 views

Regularized discriminant analysis in Matlab

I am trying to do the 2-class classification using regularized discriminant analysis in Matlab using fitdiscr() function. The coefficients are stored in the object ...
0
votes
0answers
86 views

Cutoff value in linear discriminant analysis with two groups

I have a simple linear discrimininant analysis with two classes. Prior probabilitiest are fixed to 0.5 and number of cases is equal between groups. In this case the cutoff value could be calculated as ...
0
votes
0answers
49 views

Ensemble LDA on different feature spaces?

I'm working on a classification problem where I'd like to do the following: I have a space of features that live in $R^m$, and another set of features that are related that live in $R^n$. I want to ...
0
votes
1answer
95 views

Using PCA, clustering, and LDA together

After reading about both algorithms (Principal Component analysis and Linear Discriminant analysis), I started using them combined in a way which appeared intuitive to me. I have a data set that I ...
0
votes
0answers
18 views

1D score from 3-class LDA

I am using 3-class linear discriminant analysis on a data set. The 3 class labels correspond to a single value, with high, mid and low values (labels -1, 0, and 1). The 3-class LDA works much better ...
0
votes
1answer
161 views

What methodology does proc varclus use to reduce the number of variables

In statistics, we can use methods like principal component analysis, linear discriminant analysis for variable reduction. In SAS, there is a proc called VARCLUS which is used for variable reduction. ...
0
votes
0answers
57 views

LDA - Linear discriminant function

What is the formula used to compute the posterior probability for LDA in R ? I have a unbalanced class 97% to 3%. Does LDA is good in this case ?