Hot answers tagged discriminant-analysis
7
If you mean LDA I would say the name, linear discriminant analysis, can be explained historically dating back at least to Fisher's paper from 1936, which, to the best of my knowledge, precedes the current terminology and distinction in machine learning between a discriminative and a generative model. Not that Fisher called it linear discriminant analysis ...
7
Discriminants are the axes and the latent variables which differentiate the classes most strongly. Number of possible discriminants is $min(k-1,p)$. For example, with k=3 classes in p=2 dimensional space there can exist at most 2 discriminants such as on the graph below. (Note that discriminants are not necessarily orthogonal as axes drawn in the original ...
6
The obvious "common sense" way to resolving your problem is to
Get the conclusion using the full data set. i.e. what results will you declare ignoring intermediate calculations?
Get the conclusion using the data set with said "outliers" removed. i.e. what results will you declare ignoring intermediate calculations?
Compare step 2 with step 1
If there is ...
6
While "The Elements of Statistical Learning" is a brilliant book, it requires a relatively high level of knowledge to get the most from it. There are many other resources on the web to help you to understand the topics in the book.
Lets take a very simple example of linear discriminant analysis where you want to group a set of two dimensional data points ...
5
I think that Multi-class LDA classifier always (well, in most practical tasks) out-performs 2 class LDA. And I will try to describe why.
Have a look at the example dataset:
You have three classes here. And let's say you want to build one-vs-other classifier with LDA for the blue class.
The estimated mean for class "blue" is zero, but the estimated mean ...
5
Regarding the "Real Values"
The "Real Values" are better called "confidences" or (from my pov the most common term) "scores".
Such scores are often normalized so that they sum up to 1 for all classes. They represent a measure how, well, confident the model is that the presented example belongs to a certain class. They are highly dependent on the general ...
4
I don't know of any papers on this. I've used this approach, for descriptive purposes. DFA provides a nice way to summarize group differences and dimensionality with respect to the original variables. One might more easily just profile the groups on the original variables, however, this loses the inherently multivariate nature of the clustering problem. ...
4
Here is a reference to one of Efron's papers:
http://www.jstor.org/discover/10.2307/2285453?uid=3739864&uid=2129&uid=2&uid=70&uid=4&uid=3739256&sid=47699111262887
Here is an abstract that mentions O'Neill's papers related to his Ph D dissertation:
Comparison of generative and discriminative classifiers is an
ever-lasting topic. ...
4
The idea to get a classifier for new cases after the classes had been identified in a cluster analysis is in itself natural and sane. Discriminant analysis could be such a tool, especially after K-means clustering. However, the DA's classifier will work well if assumptions of DA hold, such as (1) continuous data with approximately multivariate normal ...
4
Classification in LDA goes as follows (Bayes' approach). [About extraction of discriminants one might look here.]
According to Bayes theorem, the sought-for probability that we're dealing with class $k$ while observing currently point $x$ is $P(k|x) = P(k)*P(x|k) / P(x)$, where
$P(k)$ – unconditional (background) probability of class $k$; $P(x)$ – ...
4
You can try the lda() function in package MASS which should come with all R installations as it is a Recommended package.
Load the package via:
library("MASS")
Then read the help page for lda()
?lda
paying particular attention to the example section. When back at the prompt you can run the examples using example(lda)
3
I take it that the question is about LDA and linear (not logistic) regression.
There is a considerable and meaningful relation between linear regression and linear discriminant analysis. In case the DV consisting just of 2 groups the two analyses are actually identical. Despite that computations are different and the results - regression and discriminant ...
3
I have 3 points to say. First, what is "good" documentation for you? There is a lot of pages on DA and downloadable books on multivariate statistics (where DA is discussed) in the Internet. Some texts are superficial and easy, some are more sophisticated.
Second, because you have just 2 groups your DA will be virtually equivalent to the multiple linear ...
3
OK, since nobody answered I think that, after some experimentation, I can do it myself. Following discriminant analysis guidelines, let T be the whole cloud's (data X, of 2 variables) sscp matrix (of deviations from cloud's centre), and let W be the pooled within-cluster sscp matrix (of deviations from a cluster centre). B=T-W is the between-cluster sscp ...
3
As you say, LDA is supervised.
How does your supervisor define "learning"?
But yes, usually it is counted as supervised learning.
Reference, e.g. first 2 pages of The Elements of Statistical Learning
You can use LDA models for prediction of new cases. (I'd say that implies that something has been learned
However, you can also put emphasis on the ...
3
I believe (and someone will correct me if I'm wrong) that "classification function" can be used much more broadly - for any function that does classification, whether from logistic regression, discriminant analysis or whatever. "Discriminant function" is restricted to those from discriminant analysis.
At least, that's how I've seen them used.
2
You can get rid of some by looking for pairs that are very highly correlated and randomly deleting one of the pair.
Then you can look at partial least squares, and pick variables that are important in the PLS solution.
I did this with a similar problem and it worked pretty well (that is, the resulting discriminant function did pretty well)
2
The main difference between the coefficients and the correlations (elements of structure matrix) is not that these are less stable than those. Coefficient shows partial (i.e. unique) contribution of the variable to the discriminant function score, it is like regression coefficient. Correlation shows omnibus (i.e. unique + shared with other variables) ...
2
There are several nonparametric methods for discriminant analysis: rank methods, classifiers based on robust estimators of location and scale (M-estimators or MCD-estimators, for instance), and so on. Have you decided what kind of method that you want to use?
As for the impact of non-normality on LDA and QDA, I'd recommend that you have a look at the ...
2
The normality assumption is only a criteria for optimality in the sense that if the class conditional densities for the feature vector are normally distributed with known mean vectors and known equal covariance matrices the linear discriminant function is the Bayes rule (optimal) when the two types of errors have equal loss weights. The actual Fisher linear ...
2
The first question is whether you already know which frog belongs to which morphotype If you do know, and your goal is to use these frogs to better analyze how the morphotypes vary on these variables, then you want discriminant analysis. This might enable later investigators to accurately place frogs into morphotypes based on these variables.
If you do not ...
2
I think you are very confused about what principal component analysis is and the Chang paper has added to your confusion. First we have multivariate data in say k dimensions the principal components are a particular transformation of the coordinates such that the first principal component exhibits the largest variation in the data for any one component. ...
2
Essentially these two constraints basically require the training data to be correctly classified, and at least a certain distance from the decision threshold 0. The hyperplane that fulfils these constraints with the smallest norm of the weights will have the maximal margin. The value $\pm 1$ is essentially arbitrary, you could replace it with $\pm$ any ...
2
wx + b = -1 and wx + b = 1; These equations represent two parallel hyperplanes that are formed based on samples class (-1, +1). These two hyperplanes are used to optimize the distance between classes and to get optimal hyperplane. For optimization problem Lagrange multipliers are used.
You can find brief description on resources listed below.
Online ...
2
You may have discovered the fact that LDA models can suffer from instability: slight changes in the training set lead to very different models.
There's a whole body of literature on the variance of resampling methods for validation. This may give you a good start.
See also here for a related discussion.
(I'm sure there is a literature list somewhere ...
2
After ROC analysis we obtained a better accuracy, when we report the
accuracy of the classifier, which value we should use?
It is important to realize that ROC analysis in this case is not improving your results. The reason you are seeing better results is that you are calculating the ROC curve based on all of your data, while the LDA is calculated ...
1
If you have two class each with data distributed with densities f$_1$ and f$_2$ then the Bayes rule that minimizes the expected classification error loss function with equal loss for each error selects class 1 for observation vector x if f$_1$(x)/f$_2$(x)>1 and selects class 2 otherwise. The LDA becomes this Bayes rule under special conditions on the ...
1
I think that you know the group membership so as @PeterFlom said discriminant analysis is a good altternative. A similar method would be to estimate a Multinomial (logit or probit) model. In this model, you estimate the probability of clasyfing a frog into a given $k$ group depending on its characteristics $x$.
$P[G=k]=\Phi(\sum \beta_j^k x_j)$
where ...
1
(You probably found out by now, but in case someone else needs this:)
Centering the data is independent of the projection (LDA projects into a $n_\text{classes} - 1$ dimensional space, and it doesn't matter at all wheter this is one or more dimensions).
Generally speaking, translation (i.e. using a different center) doesn't change the predictions of an LDA ...
1
For a multivariate normal distribution the contours of constant probability fall on an ellipsoid. LDA is most often used in the context of multivariate normal features. A contour of constant probability is also the set of points in the sapce of the multivariate normal distribution that have the same Mahalanobis distance. So the greater the Mahalanobis ...
Only top voted, non community-wiki answers of a minimum length are eligible
