Linear (aka Fisher's) discriminant analysis (LDA) is a classification and a dimensionality reduction method. It finds low-dimensional subspace with the strongest class separation and uses it to perform classification.

learn more… | top users | synonyms

0
votes
0answers
34 views

How to use LDA results for prediction? And how to assess model fit?

I am trying to understand how to interpret the results I get from LDA. Running from the iris dataset in R, I can see the discriminant coefficients are in the model and then I can plot the model to ...
2
votes
1answer
49 views

Meaning of “reconstruction error” in PCA and LDA

I am implementing PCA, LDA, and Naive Bayes, for compression and classification respectively (implementing both an LDA for compression and classification). I have the code written and everything ...
3
votes
1answer
74 views

How to build a decision tree with a constraint on sensitivity?

I am trying to develop a classification model on a sample of people which will discriminate between "Type A" and "Not-Type A" people. Due to external factors, the minimum sensitivity for this ...
2
votes
0answers
18 views

DFA in SPSS: Sorts effectively, but Box's M is still 0.000. Is the analysis worthless?

I am a geologist attempting to apply the discriminant function analysis to surface features I have mapped in ArcGIS. At the moment I have 4 dimensionless sorting variables calculated for each feature, ...
1
vote
1answer
95 views

Variable selection using cross-validated PLS model when permutation test shows lack of significance

I understand that the permutation test on PLS can help to detect overfitting of the PLS model. Usually if the p-value is greater than a criterion, say 0.05, it means that the model is overfitting and ...
0
votes
2answers
71 views

choosing between logistic and discriminant

I am looking at regularized logistic regression, (l1 and l2 at the moment) and regularized discriminant analysis. How do I compare the two? I was thinking of doing gcv on both methods over a set of ...
2
votes
0answers
64 views

Why does discriminant analysis prevent us from finding more than $K-1$ linear 'features' and what does it mean?

According to Bishop's Machine Learning and Pattern Recognition, the cost function for linear discriminant analysis (LDA) with $K>2$ classes is $$J(\mathbf w) = \mathrm{Tr}\left\{\left(\mathbf W ...
1
vote
0answers
38 views

Overfitting with diagonal matrix of covariances

Sample is a realisation of vector of random variables, each variable ~ $\mathcal N$. The number of samples is small, the length of sample is big. I would like to perform one-class LDA (linear ...
1
vote
0answers
36 views

Train & predict probabilities using LDA having multiple collinearities

I am trying to fit an LDA model and predict conditional probabilities of class membership with it. I believe I understand the basic method to do this using the covariance matrix and class means, but ...
5
votes
1answer
82 views

Seeming disagreement between learning sources about linear/quadratic and Fisher's discriminant analysis

I'm studying discriminant analysis, but I'm having a difficult time reconciling several different explanations. I believe I must be missing something, because I've never encountered this (seeming) ...
3
votes
0answers
28 views

What happens to linear discriminant analysis when $p>n$?

I have a general question regarding LDA (Fisher's linear discriminant analysis). What happens if the sample size $n$ is smaller than the dimensionality $p$ (number of predictors)? Is it possible to ...
0
votes
0answers
33 views

Linear Discriminant Analysis for dimensionality reduction - choosing the dimension

I'm using Linear Discriminant Analysis to do dimensionality reduction of a multi-class data. What is the best method to determine the "correct" number of dimensions? Can I use a method similar to PCA, ...
0
votes
0answers
18 views

How to see a Gaussian Discriminant Analysis (GDA) as a linear model for multiclass case?

In GDA we can assume that posterior probability for each of $K$ possible classes is Gaussian with same variance $\Sigma$, and different means $\mu_k$, ie. $$p(\mathbf x|C_k):\mathcal N(\mathbf ...
4
votes
2answers
194 views

Discriminant analysis vs logistic regression

I found some pros of discriminant analysis and I've got questions about them. So: When the classes are well-separated, the parameter estimates for logistic regression are surprisingly unstable. ...
1
vote
0answers
33 views

How generate data for classification tasks in R

At the last time i asked, is it possible in R to generate data on the given R -square, and as it turns out it's simple. ...
1
vote
0answers
30 views

How to calculate mahalanobis distance in SIMCA where different number of PCs are obtained for each class

I am working on a software that does SIMCA using mahalanobis distances with the following steps(excluding the classification of new objects for simplicity): Center each class individually Apply PCA ...
0
votes
0answers
23 views

Transformation to be used for continuous variable

I have a data set where I am doing a binary classification. I have close to 500 features and 200K observations. Now I also have few continuous variables as features. I don't think just using these ...
1
vote
1answer
74 views

Regularized discriminant analysis in Matlab

I am trying to do the 2-class classification using regularized discriminant analysis in Matlab using fitdiscr() function. The coefficients are stored in the object ...
0
votes
0answers
43 views

Cutoff value in linear discriminant analysis with two groups

I have a simple linear discrimininant analysis with two classes. Prior probabilitiest are fixed to 0.5 and number of cases is equal between groups. In this case the cutoff value could be calculated as ...
0
votes
0answers
40 views

Ensemble LDA on different feature spaces?

I'm working on a classification problem where I'd like to do the following: I have a space of features that live in $R^m$, and another set of features that are related that live in $R^n$. I want to ...
0
votes
1answer
66 views

Using PCA, clustering, and LDA together

After reading about both algorithms (Principal Component analysis and Linear Discriminant analysis), I started using them combined in a way which appeared intuitive to me. I have a data set that I ...
0
votes
0answers
13 views

1D score from 3-class LDA

I am using 3-class linear discriminant analysis on a data set. The 3 class labels correspond to a single value, with high, mid and low values (labels -1, 0, and 1). The 3-class LDA works much better ...
0
votes
1answer
59 views

What methodology does proc varclus use to reduce the number of variables

In statistics, we can use methods like principal component analysis, linear discriminant analysis for variable reduction. In SAS, there is a proc called VARCLUS which is used for variable reduction. ...
0
votes
0answers
41 views

LDA - Linear discriminant function

What is the formula used to compute the posterior probability for LDA in R ? I have a unbalanced class 97% to 3%. Does LDA is good in this case ?
2
votes
0answers
85 views

How to use LDA results for feature selection?

I am working on the Forest type mapping dataset which is available in the UCI machine learning repository. I have 27 features to predict the 4 types of forest. I am performing a Linear Discriminant ...
2
votes
1answer
46 views

What learning occurs in linear discriminant analysis?

From what I understand, linear discriminant analysis (LDA) has an objective function, where you try to find a matrix that maps data from a $p$-dimensional feature space to a $r$-dimensional feature ...
1
vote
0answers
17 views

Appropriateness of LDA for classification when majority of features are binary?

Am I correct in understanding that when using linear discriminant analysis, one assumes that the feature values are normally distributed? If so, does this mean LDA is very poorly suited for data ...
2
votes
0answers
44 views

Separation of points clouds via classification methods

I have multiple images from a 3D-Scanner in point cloud form. Part of the image is a fixture to hold the object to be scanned. I want to extract the object itself by classifying the fixture and the ...
0
votes
0answers
11 views

I want to identify what combination of discrete variables best classify into one of three groups

I have a series of images for patients who fall within three distinct diagnoses we have scored multiple features (about 8-10) for each patients into yes present or no not present how do i identify ...
1
vote
1answer
79 views

Conceptual undersanding of linear discriminant analysis

Can someone explain to a newbie the concepts of linear discriminant analysis? I am not looking for a technical implementation like this. I wish to understand it conceptually. I understand logistic ...
0
votes
1answer
67 views

Data normalization prior to PCA? [duplicate]

I want to get some intuition on normalization prior to feature selection with PCA. I'm sure z-normalization is a bad idea, since it normalizes the variances to 1 for each feature, PCA will be ...
1
vote
0answers
61 views

When is the scatter matrix in linear discriminant analysis singular?

In linear discriminant analysis (LDA), when there are fewer data instances than the number of dimensions (i.e., when the data matrix is of order $n \times m$ where $n$ is less than $m$), the ...
3
votes
1answer
166 views

How LDA, a classification technique, also serves as dimensionality reduction technique like PCA

In this article , the author links linear discriminant analysis (LDA) to principal component analysis (PCA). With my limited knowledge, I am not able to follow how LDA can be somewhat similar to PCA. ...
0
votes
0answers
15 views

Structural partition for cross validation

Can the partitioning of data for cross validation be used structurally to assess the transfer ability of an effect of one factor across another factor? A MANOVA performed on multivariate data reveals ...
0
votes
1answer
90 views

Fisher Projection vs Linear Discriminant Analysis [closed]

Basically, I am confused between Fisher and LDA. Looking for differences between the two. How is the Fischer projection computed in R?
1
vote
1answer
164 views

Difference between within-group and between-group covariance matrices in linear discriminant analysis

Could someone explain to me the difference between within-group covariance matrix and between-group covariance matrix in the context of linear discriminant analysis?
0
votes
0answers
19 views

Sphering and Centering Data

If I have a bunch of data points that like in p-dimensions, what does it mean if the data is sphered and centered. I never really understood this concept and was wondering if anyone could explain it ...
2
votes
1answer
175 views

Why are discriminant analysis results in R (lda) and SPSS different?: Constant term

I tried discriminant analysis with lda() in R and in SPSS, but the scalings were different, why? N, how to get (Constant) with R like SPSS result? data: ...
2
votes
2answers
299 views

Best way to bin continuous data

I have a data frame with 1 vector of integers and 1 as a character factor like so: I have created a linear model that shows a relationship between age and party affiliation. I now want to determine ...
1
vote
0answers
20 views

Multivariate technique to determine whether an effect is intermediate to two or more other effects?

Let’s say I am looking at soil microbial community structure. I have a multivariate dataset with 20 dependent variables indicating various soil microbial biomarkers in the soil. My treatments include ...
1
vote
1answer
59 views

A book on discriminant analysis

Can anyone suggest a good book on discriminant analysis - comprehensible and detailed? (Kendall and Stuart write about the subject too concisely.)
0
votes
0answers
69 views

LDA scores too big

I'm trying to do dimensionality reduction with linear discriminant analysis (LDA) in MATLAB. I'm using this code to calculate the coefficients. But I'm confused whether (and when) should I center the ...
0
votes
0answers
69 views

Data sets for which PCA can classify better than LDA (using a very small training set)

Can you provide an example of a dataset where PCA can find better discriminant directions than (LDA) Linear Discriminant Analysis? One example is UCI's wine data set. If you use only 2 observations ...
2
votes
0answers
56 views

Fisher LDA is a Bayes Classifier?

I've been going over many material in classification algorithms, and it seems that under the constraint that the covariance matrices are the same for a two-class problem then classifying a vector $x$ ...
0
votes
0answers
25 views

Non-negative Fisher LDA?

Is any one aware of any existing publication on non-negative solution for (Fisher's) linear discriminant analysis?. In particular: $$\text{maximize}_{w\geq0} ~ ...
0
votes
0answers
75 views

How to quantify performance of Linear Discriminant Analysis (LDA)?

I have implemented Linear Discriminant Analysis (LDA) for dimensionality reduction in C. But I don't know how to quantify performance of the LDA. Could someone help me?
1
vote
1answer
51 views

pattern classification when the prior probabilities are not equal

In the case of 2 class classification, the decision boundary occurs when 2 discriminant functions are equal: $$ g_1(x) = g_2(x) $$ $$ g_i(x) = p(x|w_i)P(w_i) $$ $$ p(x|w_i) = ...
4
votes
1answer
205 views

Supervised dimensionality reduction

I have a data set consisting of 15K labeled samples (of 10 groups). I want to apply dimensionality reduction into 2 dimensions, that would take into consideration the knowledge of the labels. When I ...
0
votes
1answer
19 views

What metric should be used to compare models when the prediction is a probability of an event occurring?

Let's say that I am trying to predict whether or not it will rain today, and I build a model that gives me a % chance of the event occurring. Now let's say I build another model, and I want to see ...
1
vote
0answers
210 views

Interpreting LD1 and LD2 in lda in R

I'm conducting an experiment in R. I am using the rattle library that contains a sample of the wine related data. Within this ...