# Tagged Questions

Discriminant (function) analysis (DA) is a dimensionality reduction and classification method. It finds low-dimensional subspace with the strongest class separation and uses it to perform classification. Most well-known is Linear Discriminant analysis (LDA) which provides linear borders of ...

25 views

### Why does executing a PCA on my unbalanced dataset change the prediction distribution of my test so much?

I'm using prtools to try and solve a machine learning problem and trying to use a PCA to help me with this. Now I have a dataset with 13 features (columns), 10000 ...
17 views

### Compute a Quadratic discriminant analysis (QDA) in R assuming not normal data and missing information

In this course, the professor is saying that we can compute a QDA with missing data points and non-normal data (even if this assumption can be violated). But the problem is that I don't know any ...
32 views

### Confusion Matrix values of zero

My question is almost exaclty the same as this one: Suspicious Amount of Zeros in Confusion Matrix, but no one seems to have responded so I figure I'll ask again. I've got a LOO confusion matrix that ...
35 views

### Using factor scores in discriminant analysis

I'm looking at indicators of earnings fraud in publicly-held corporations. My classification is dichotomous, Yes, company is likely to engage in EF, or no, it's not. I have 15 financial variables ...
39 views

### Deriving the overall covariance matrix in LDA given the within-group one and all class means

I'm stuck at this simple excercise.
40 views

### Are PLS-DA and PLS-LDA the same?

Seems like a trivial question but one for which I can't seem to find an answer. Are PLS-DA (partial least squares discriminant analysis) and PLS-LDA (partial least squares followed by linear ...
50 views

### Unstable projection in LDA space in $n<p$ situation [duplicate]

I'm trying to classify (LDA) few samples (n=12) in a high dimensional feature space (p=24) into 3 classes. First I reduced the ...
23 views

### Does Gaussian discriminant analysis and linear discriminant analysis refer the same algorithm?

I'm pretty new to LDA and I came across other terminology called Gaussian discriminant analysis elsewhere. Since LDA assumes the normality or normal distribution of the data which is same as ...
15 views

### Checking Multivariate Gaussian (normal) distribution before applying LDA

I have a data set with multiple predictors. I want to apply Linear Discriminant analysis (LDA) on my data. But before doing that I must confirm that my data is from multivariate Gaussian Distribution ....
9 views

### How are the between and within sum-of-squares matrices defined?

I was reading Tibishirani's paper, and on page 2 I came across the terms between and within sum-of-squares matrices. How are those matrices defined? Are they related to the Uncorrected Sums of Squares ...
34 views

### Common covariance matrix in linear discriminant analysis

Say I want to perform LDA classification involving three classes with within-class covariance matrices $$\hat{\Sigma}_1 \,, \hat{\Sigma}_2 \, , \hat{\Sigma}_3$$ and that these matrices are calculated ...
52 views

51 views

### Comparison of LDA vs KNN time complexity

Which algorithm has a better performance in terms of time complexity, LDA or KNN?
35 views

### Two-classes LDA on third class

I am trying to implement a $N$ classes classification with several 2-classes LDAs. I actually am using LDA as a projection method instead of classification, so it might be more a factor analysis. If ...
16 views

### How do I derive the Discriminant Function in Linear Discriminant Analysis

From An Introduction to Statistical Learning with Applications in R on page 143, the authors talk about obtaining the discriminant function in the case of LDA for >1 predictors. Assuming that we are ...
10 views

### Link between the FDA and LS-SVM

I am reading tutorial written by Johan Suykens:Least Squares Support Vector Machines On page 19,he mentions link with kernel Fisher Discriminant Analysis Project ...
30 views

### LDA and QDA main concepts

I've been studying LDA and QDA main concepts and already can implement those algorithms, but I'm not good at understanding the theory of this technique. Textbooks and Google gives me some properties ...
41 views

### Unsupervised clustering based on discriminant line

I have quite specific statistical problem, highly limited by its ecological interpretation. I have plenty of "time series" data - I need to link supression of photosythesis to the lack of light (PAR) ...
8 views

### how to define which of measured, non gaussian variables are effective in discriminate given groups?

I have eight groups (100 samples each) and 43 evaluated variables, with different distributions (some of them are right skewed, some left skewed, some with many zeros etc.). I'm trying to understand ...
39 views

### Difference between GMM classification and QDA

I know that every class has the same covariance matrix $\Sigma$ in linear discriminant analysis (LDA), and in quadratic discriminant analysis (QDA) they are different. When using gaussian mixture ...
58 views

### Interpretation of the cluster criterion $\operatorname{tr}(S_W^{-1}S_B)$

There is a cluster criterion defined as: $$\mathcal{C} = \operatorname{tr}(S_W^{-1}S_B) = \sum_{i=1}^d \lambda_i,$$ where $\operatorname{tr}$ is the trace, $S_W$ is the pooled within-group scatter ...
81 views

### LDA for dimensionality reduction usage

I have a original dataset with 70 samples, each sample with 96 features. The samples are labeled as 0 or 1. So I use linear discriminant analysis (LDA) to reduce the dimensionality of all the dataset, ...
75 views

### Using projected points from Linear Discriminant Analysis to generate probability density function

I am using Linear Discriminant Analysis technique to get the best possible separation between two distributions. I am using R for my programming. For LDA, I find the between-class scatter matrix(B) ...
83 views

42 views

### Overfitting with diagonal matrix of covariances

Sample is a realisation of vector of random variables, each variable ~ $\mathcal N$. The number of samples is small, the length of sample is big. I would like to perform one-class LDA (linear ...
43 views

### Train & predict probabilities using LDA having multiple collinearities

I am trying to fit an LDA model and predict conditional probabilities of class membership with it. I believe I understand the basic method to do this using the covariance matrix and class means, but ...
188 views

### Seeming disagreement between learning sources about linear/quadratic and Fisher's discriminant analysis

I'm studying discriminant analysis, but I'm having a difficult time reconciling several different explanations. I believe I must be missing something, because I've never encountered this (seeming) ...
42 views

### What happens to linear discriminant analysis when $p>n$?

I have a general question regarding LDA (Fisher's linear discriminant analysis). What happens if the sample size $n$ is smaller than the dimensionality $p$ (number of predictors)? Is it possible to ...
82 views

### Linear Discriminant Analysis for dimensionality reduction - choosing the dimension

I'm using Linear Discriminant Analysis to do dimensionality reduction of a multi-class data. What is the best method to determine the "correct" number of dimensions? Can I use a method similar to PCA, ...
28 views

### How to see a Gaussian Discriminant Analysis (GDA) as a linear model for multiclass case?

In GDA we can assume that posterior probability for each of $K$ possible classes is Gaussian with same variance $\Sigma$, and different means $\mu_k$, ie. p(\mathbf x|C_k):\mathcal N(\mathbf \mu_k,\...
456 views

### Discriminant analysis vs logistic regression

I found some pros of discriminant analysis and I've got questions about them. So: When the classes are well-separated, the parameter estimates for logistic regression are surprisingly unstable. ...
38 views

### How generate data for classification tasks in R

At the last time i asked, is it possible in R to generate data on the given R -square, and as it turns out it's simple. http://stackoverflow.com/questions/33920813/how-to-generate-data-on-the-a-given-...
76 views

### How to calculate mahalanobis distance in SIMCA where different number of PCs are obtained for each class

I am working on a software that does SIMCA using mahalanobis distances with the following steps(excluding the classification of new objects for simplicity): Center each class individually Apply PCA ...
40 views

### Transformation to be used for continuous variable

I have a data set where I am doing a binary classification. I have close to 500 features and 200K observations. Now I also have few continuous variables as features. I don't think just using these ...
138 views

### Regularized discriminant analysis in Matlab

I am trying to do the 2-class classification using regularized discriminant analysis in Matlab using fitdiscr() function. The coefficients are stored in the object ...
92 views

### Cutoff value in linear discriminant analysis with two groups

I have a simple linear discrimininant analysis with two classes. Prior probabilitiest are fixed to 0.5 and number of cases is equal between groups. In this case the cutoff value could be calculated as ...
50 views

### Ensemble LDA on different feature spaces?

I'm working on a classification problem where I'd like to do the following: I have a space of features that live in $R^m$, and another set of features that are related that live in $R^n$. I want to ...
99 views

### Using PCA, clustering, and LDA together

After reading about both algorithms (Principal Component analysis and Linear Discriminant analysis), I started using them combined in a way which appeared intuitive to me. I have a data set that I ...
19 views

### 1D score from 3-class LDA

I am using 3-class linear discriminant analysis on a data set. The 3 class labels correspond to a single value, with high, mid and low values (labels -1, 0, and 1). The 3-class LDA works much better ...
187 views

### What methodology does proc varclus use to reduce the number of variables

In statistics, we can use methods like principal component analysis, linear discriminant analysis for variable reduction. In SAS, there is a proc called VARCLUS which is used for variable reduction. ...
61 views

### LDA - Linear discriminant function

What is the formula used to compute the posterior probability for LDA in R ? I have a unbalanced class 97% to 3%. Does LDA is good in this case ?
127 views

### How to use LDA results for feature selection?

I am working on the Forest type mapping dataset which is available in the UCI machine learning repository. I have 27 features to predict the 4 types of forest. I am performing a Linear Discriminant ...