Given multivariate data split into several subsamples (classes) the analysis finds linear combinations of variables, called discriminant functions, which discriminate between classes and are uncorrelated. The functions are applied then to assign old or new observations to the classes. Discriminant ...
9
votes
1answer
620 views
Treatment of outliers produced by Kurtosis
I was wondering if anyone could help me with information about Kurtosis (i.e. is there any way to transform your data to reduce it?)
I have a questionnaire dataset with a large number of cases and ...
8
votes
1answer
677 views
Cluster Analysis followed by Discriminant Analysis
What is the rationale, if any, to use Discriminant Analysis (DA) on the results of a clustering algorithm like k-means, as I see it from time to time in the literature (essentially on clinical ...
7
votes
2answers
431 views
Why prediction of a predicted variable from a discriminant analysis is imperfect
I am puzzled by something I found using Linear Discriminant Analysis. Here is the problem - I first ran the Discriminant analysis using 20 or so independent variables to predict 5 segments. Among the ...
6
votes
2answers
592 views
How does linear discriminant analysis reduce the dimensions?
There are words from "The Elements of Statistical Learning" on page 91:
The K centroids in p-dimensional input space span at most K-1 dimensional subspace, and if p
is much larger than K, this ...
5
votes
1answer
157 views
Variant of discriminant analysis for known multiple independent classifications?
I have a large data set: over 100,000 data points, each with 60 dimensions. I want to display the data in 2D to visibly maximize the separation between classes, which I know for each point. I asked a ...
5
votes
1answer
217 views
Classifying clusters using discriminant analysis
Suppose I've data for 100 individuals for 5 variables, say Var1, Var2,...Var5. I run the cluster analysis using these 5 variables on these 100 rows & got 3 clusters. Now, I want to differentiate ...
4
votes
3answers
1k views
What is the relationship between regression and linear discriminant analysis?
Is their a relationship between regression and linear discriminant analysis? What are their similarities and differences? Does it make any difference for two classes and more than two classes?
4
votes
1answer
405 views
Multi class LDA vs 2 class LDA
The problem of designing a multi-class classifier using LDA can be expressed as a 2 class problem(one vs everything else) or a multi-class problem.
Why is it that in certain cases Multi-class LDA ...
4
votes
2answers
162 views
How do you identify the variables that separate several groups?
I don't have much background on statistics. I am working on multivariate morphometrics of a sample of frogs. I have a data matrix of 19 variables (continuous characteristics) for around 250 samples. ...
4
votes
0answers
134 views
Using QDA for Non-Gaussian distributions
I am evaluating a Quadratic Discriminant Analysis (QDA) classifier on a high-dimensionality feature set. The features come from highly non-Gaussian distributions. However, when I transform the ...
3
votes
2answers
567 views
Why are Gaussian “discriminant” analysis models called so?
Gaussian discriminant analysis models learn $P(x|y)$ and then apply Bayes rule to evaluate
$$P(y|x) = \frac{P(x|y)P_{prior}(y)}{\Sigma_{g \in Y} P(x|g) P_{prior}(g) }.$$ Hence, they are generative ...
3
votes
1answer
209 views
What do “real values” refer to in supervised classification?
I'm using supervised classification algorithms from mlpy to classify things into two groups for a question-answering system. I don't really know how these algorithms work, but they seem to be doing ...
3
votes
2answers
488 views
Linear discriminant analysis and Bayes rule
What is the relation between Linear discriminant analysis and Bayes rule? I understand that LDA is used in classification by trying to minimize the ratio of within group variance and between group ...
3
votes
1answer
84 views
Is discriminant analysis supervised learning?
Is linear discriminant analysis, specifically Linear Programming Discriminant Analysis (LPDA), supervised learning? Can you provide a valid reference that states so if possible.
My study supervisor ...
3
votes
2answers
667 views
How to interpret a predictor with a positive structure coefficient and a negative standardised coefficient in discriminant function analysis?
I am doing a discriminant function analysis and I have four continous independent variables
and one categorical dependent variable (that has 3 groups). I have
chosen to do this analysis to see how ...
3
votes
1answer
260 views
Plotting a discriminant as line on scatterplot
Given a data scatterplot I can plot the data's principal components on it, as axes tiled with points which are principal components scores. You can see an example plot with the cloud (consisting of 2 ...
3
votes
1answer
1k views
Deriving total (within class + between class) scatter matrix
I was fiddling with PCA and LDA methods and I am stuck at a point, I have a feeling that it is so simple that I can't see it.
Within-class ($S_W$) and between-class ($S_B$) scatter matrices are ...
3
votes
2answers
312 views
Theory on discriminant analysis in small sample size conditions
I see a similarity between a problem I'm working on and Linear (or Quadratic) Discriminant Analysis when the sample size is smaller than $p+1$.
I'm interested in theory bounding the generalization ...
3
votes
1answer
166 views
Choosing variables for Discriminant Analysis
I've 110 variables & 200 data points. Of this 110 variables, one is group variable (say "brown eye","blue eye"). I want to use discriminant analysis to classify the groups based on remaining 119 ...
3
votes
0answers
129 views
LDA, Significance of orthonormality- Trace Ratio Maximization
The objective of fisher linear discriminant analysis can be formulated as maximizing $\frac{Tr[X^TAX]}{Tr[X^TBX]}$ over $X$ where $A$ and $B$ are positive semi-definite with orthonormality constraints ...
2
votes
2answers
125 views
Where does the definition of the hyperplane in a simple SVM come from?
I'm trying to figure out support vector machines using this resource. On page 2 it is stated that for linearly separable data the SVM problem is to select a hyperplane such that $\vec{x}_i\vec{w} + b ...
2
votes
2answers
191 views
Can you use discriminant analysis to classify new observations into categories generated by a previous $k$-means clustering?
After doing k-means clustering on a set of observations, I would like to construct a discriminant function so as to classify new observations into the categories I found after k-means. Is this at all ...
2
votes
1answer
149 views
What exactly the ROC curve can tell us or can be inferred?
(I post this originally at http://stackoverflow.com/questions/15477282/what-exactly-the-roc-curve-can-tell-us-or-can-be-inferred, but people directed me to here. Sorry about posting this twice.)
I ...
2
votes
3answers
243 views
Does PCA followed by LDA make sense?
This is a question about classification. I am a neuroscience student with little experience of classification methods and I'd be grateful for any advice about the best way to implement a linear ...
2
votes
1answer
306 views
Non-parametric discriminant analysis in R
I want to use Discriminant Analysis between two non normal populations in R. Can anybody tell me the name of the R function to do so?
Could also anybody tell me how accurate my results will be if I ...
2
votes
3answers
301 views
Usage of LDA with more than two classes
I'm reading about the Linear Discriminant Analysis by Fisher and I have a couple of questions about its usage.
If you have k>2 classes in a two-dimensional space you find k−1 vectors that you need ...
2
votes
1answer
121 views
Fisher discrimination power of a variable and Discriminant analysis
Apparently,
the Fisher analysis aims at simultaneously maximising the
between-class separation, while minimising the within-class
dispersion. A useful measure of the discrimination power of a ...
2
votes
1answer
222 views
Decision boundaries from coefficients of linear discriminants?
I have a data set with four variables and 3000+ observations on which I performed an LDA. I was wondering how I can use the scaled coefficients of linear discriminants (output of R shown below as ...
2
votes
1answer
154 views
How to estimate the deposit mix of a bank using interest rate as the independent variable?
Let's say a bank has 5 different types of deposits. One type is certificates of deposits (CD), and the other 4 types are different checking and savings account products with various interest rates ...
2
votes
0answers
85 views
How do you detect if a given dataset has multivariate normal distribution?
I'm looking at Fisher's LDA on various datasets on UCI ML repository and trying to see where LDA might perform badly. One reason I can think of is if the data distribution is not a multi-variate ...
1
vote
1answer
102 views
Sample size and documentation for discriminant analysis
Does anybody have good documentation for discriminant analysis? I have 9 variables (measurements), 60 patients and my outcome is good surgery, bad surgery. Also, is my sample size too small?
Thank ...
1
vote
1answer
42 views
Inconsistency in cross-validation results
I have a set of dataset recorded from subjects as they perform some particular cognitive task. The data consists of 16 channels and a number of sample points per channel and I want to classify this ...
1
vote
1answer
184 views
Dealing with high dimension in principal component analysis
For very extreme high dimensions in PCA, the number of dimensions $p$ is larger than the sample size $N$, does PCA work well or does it work at all? By 'work' I mean
does it work mathematically
If ...
1
vote
1answer
122 views
How do I test whether I can properly apply LDA?
I have some data which works nicely with JMP's canned linear discriminant analysis (LDA), but after reading about LDA I'm not sure if the analysis is valid. The Wiki article notes a fundamental ...
1
vote
1answer
40 views
Why is there a sharp elbow in my ROC curves?
I have some EEG data sets that I am testing against two classes. I can get a decent error rate from LDA (the class-conditional distributions aren't Gaussian, but have similar tails and good enough ...
1
vote
1answer
170 views
How does Fisher LDA work?
Intuitively, how does Fisher LDA work? From this Linear discriminant analysis and Bayes rule I completely understood the Bayesian approach but I'm not able to relate it to the Fisher's one described ...
1
vote
1answer
182 views
LDA projection for classification
I am dealing with 2 class LDA classification problem.
During a test phase (after training), I'm trying to project a feature vector to lower dimensional space.
How do we get the projected test ...
1
vote
0answers
58 views
Using linear discriminant analysis to validate the cluster groups resulting from kmeans
I'm currently working on a cluster analysis project and ran kmeans on the data for k=2.
I was reading similar articles on similar experiments, and the investigators used discriminant analysis to ...
1
vote
0answers
73 views
How to interpret this output in Dimension Reduction?
I'm running Linear Discriminant Analysis on a dataset and then performing clustering on it. I'm reducing it to dimensions 2,6,10. On comparing metrics like Accuracy and Normalized Mutual Information, ...
1
vote
0answers
83 views
How to overcome singularity problem in Linear Discriminant Analysis?
I've code for LDA which is failing as the matrices passed to calculate eigen values are not singular and hence lead to infinite eigen values? Can anyone recommend what can be done to fix this? I've ...
1
vote
0answers
47 views
Overfitting a linear Linear Discriminant Function
I am estimating a Linear Discriminant function with 250 input variables over 4000 data records. Should I consider feature selection, am I over fitting the model? How do I know when feature selection ...
1
vote
0answers
88 views
Discriminant analysis with random effects
Is it possible to do discriminant analysis with random effects?
Is there an R package for this?
Context:
I have habitat use data for two species of frogs from radio telemetry, but nested within ...
1
vote
0answers
93 views
Different approaches of linear discriminant analysis
What are the differences in three different approaches of classification of LDA for two groups?
The three are:
1. Fisher's approach
2. regression approach
3. Bayes's approach
They will be exactly ...
1
vote
0answers
196 views
How to combine features extracted by PCA, LDA and LBP?
What I'm thinking is to combine PCA features, LDA features and LBP features together to get a higher accuracy, since I think the three features are all kind of histogram vectors and when we decide the ...
1
vote
0answers
127 views
LDA with a categorical predictor variable
I have a dataset with a categorical treatment (1 if treated, 0 if control) and many columns (34) of response variables. Each column represents a species and its response (some measured abundance) to ...
1
vote
0answers
55 views
Difference and connection between generative learning, discriminative learning and max-margin learning
I once heard that, generative learning, discriminative learning and max-margin learning can be separated in terms of their respective definition of loss function. I am not sure how to achieve that?
1
vote
0answers
359 views
How to classify new cases in discriminant analysis exactly as SPSS does?
What I have:
There is one base with already classified cases. There are 23 independent variables that were used in this classification and 10 groups.
Another base has new unclassified cases. There ...
1
vote
0answers
106 views
Range of standardized coefficients in a discriminant analysis
I want to run a discriminant analysis on different motion capture measures to see which of the measures distinguishes best between my two conditions. The problem is that some of the standardized ...
1
vote
2answers
126 views
How can you make linear discriminant analysis reduce dimensions to the number of dimensions you are looking for?
Let's say I have a $m \times n$ matrix where $m$ is the number of points and $n$ is the number of dimensions. I would like to give a target dimension parameter which is let's say d. d can be a set of ...
0
votes
1answer
60 views
Is “discriminant function” a synonym for “classification function”
"discriminant function" and "classification function" are two terms used in literature to denote a a function that maps a feature vector into a discrete class variable.
I presume "discriminant ...
