Given multivariate data split into several subsamples (classes) the analysis finds linear combinations of variables, called discriminant functions, which discriminate between classes and are uncorrelated. The functions are applied then to assign old or new observations to the classes. Discriminant ...

learn more… | top users | synonyms

2
votes
0answers
32 views

Fisher LDA is a Bayes Classifier?

I've been going over many material in classification algorithms, and it seems that under the constraint that the covariance matrices are the same for a two-class problem then classifying a vector $x$ ...
0
votes
0answers
11 views

Non-negative Fisher LDA?

Is any one aware of any existing publication on non-negative solution for (Fisher's) linear discriminant analysis?. In particular: $$\text{maximize}_{w\geq0} ~ ...
0
votes
0answers
41 views

How to quantify performance of Linear Discriminant Analysis (LDA)?

I have implemented Linear Discriminant Analysis (LDA) for dimensionality reduction in C. But I don't know how to quantify performance of the LDA. Could someone help me?
1
vote
1answer
34 views

pattern classification when the prior probabilities are not equal

In the case of 2 class classification, the decision boundary occurs when 2 discriminant functions are equal: $$ g_1(x) = g_2(x) $$ $$ g_i(x) = p(x|w_i)P(w_i) $$ $$ p(x|w_i) = ...
2
votes
1answer
50 views

Supervised dimensionality reduction

I have a data set consisting of 15K labeled samples (of 10 groups). I want to apply dimensionality reduction into 2 dimensions, that would take into consideration the knowledge of the labels. When I ...
0
votes
1answer
13 views

What metric should be used to compare models when the prediction is a probability of an event occurring?

Let's say that I am trying to predict whether or not it will rain today, and I build a model that gives me a % chance of the event occurring. Now let's say I build another model, and I want to see ...
1
vote
0answers
44 views

Interpreting LD1 and LD2 in lda in R

I'm conducting an experiment in R. I am using the rattle library that contains a sample of the wine related data. Within this ...
1
vote
1answer
39 views

plot LDA fit using R function plot()

I am doing the lab section: classifying the stock data using LDA in the book "Introduction to Statistical Learning with Applications in R", here is the lab video. Basically, this lab uses LDA to ...
1
vote
0answers
12 views

Classifying after solving discriminant function

I have a set of data (7 variables) that are used as input to ultimately making a decision to approve or not approve something. I've generated two discriminant functions that I'm happy with - one for ...
0
votes
0answers
17 views

Bagging Improvement for classification problems

Breiman proved that bagging improves the accuracy of regression. How could this be proven for a classifier model?
1
vote
0answers
16 views

Fisher kernel or KL divergence?

I am currently reading about pLSA and LDA and how can I apply these methods on calculating document similarity. I got a feeling that common similarity measure used in pLSA is Fisher kernel, but for ...
1
vote
0answers
25 views

Testing significance of linear discriminants

I have run a discriminant analysis and would like to test the significance of each resulting discriminant function - i.e., does each discriminant function contribute to the separability of the groups? ...
1
vote
0answers
25 views

How to understand “MANOVA” and multiple discriminant analysis are special cases of canonical correlation?

I am reading a statistical book which writes, Canonical correlation represents one way in which we can examine the relationship between multiple dependent variables ($\bf{Y}$) and multiple ...
0
votes
0answers
80 views

Caret feature selection [RFE] yields different features depending on reference level of two-class dependent measure

I'm using RFE from the caret package in R to select variables to be used in a linear discriminant analysis. The outcome is a binary factor, but depending on which level of the factor is used as the ...
0
votes
0answers
28 views

Why can I use the posterior probability of a classifier as a new classifier?

I have read that, when doing discriminant analysis, you can use the posterior probability you obtain using your classifier as a new fine-tuned classifier. Can anyone talk me through the rationale of ...
0
votes
0answers
30 views

Solution of Fisher LDA

I'm trying to understand the Fishers LDA. In the book I am using there is explained that one has to maximize expression $$\frac{\left( a^T\cdot \bar{x}_2-a^T\cdot \bar{x}_1 \right)^2}{a^T\cdot ...
0
votes
0answers
83 views

Obtaining significance for variables in a linear discriminant function analysis

I have run a linear discriminant function analysis using the lda() function in the MASS library to determine which of 6 ...
4
votes
1answer
72 views

Using kernels with Fisher's linear discriminant analysis

I am a bit stuck implementing the Kernel Fisher Discriminant. $$ J(\mathbf{w}) = \frac{\mathbf{w}^{\text{T}}\mathbf{S}_B^{\phi}\mathbf{w}}{\mathbf{w}^{\text{T}}\mathbf{S}_W^{\phi}\mathbf{w}} $$ $$ ...
0
votes
0answers
10 views

Analogies between Gaussian Discriminant Analysis and Joint Probability Distribution

Can I say GDA is like a full joint probability distribution over all the feature random variables? I mean, if we are given some random variables, we try to inject some conditional independencies, and ...
2
votes
0answers
37 views

Sample lower bound for binary classification in Linear Discriminant Analysis?

Below is a description of this problem: Suppose the label $Y\in\{1,0\}$ in binary classification satisfies $\Pr[Y=1]=\Pr[Y=0]=\frac{1}{2}$, and $p(X|Y=1)=\mathcal{N}(\mu_1,\Sigma)$, ...
11
votes
2answers
373 views

How to calculate Fisher criterion weights?

I am studying pattern recognition and machine learning, and I ran into the following question. Consider a two-class classification problem with equal prior class probability $$P(D_1)=P(D_2)= ...
0
votes
0answers
101 views

Plotting QDA projections in R

When doing discriminant analysis using LDA or PCA it is straightforward to plot the projections of the data points by using the two strongest factors. This can be done in R by using the x component ...
0
votes
0answers
32 views

What is the heuristic to decide number of components for LDA dimensionality reduction?

In the PCA case, I prefer to plot the variance and choose number of components regarding that plot's breaking point. In the LDA (linear disriminant analysis) case, what can be used for such an ...
0
votes
0answers
15 views

Different coefficients of linear discriminants with the same raw data

I have just tried two ways to perform a linear discriminant analysis. Mode 1 This is the data, divided in two tables (printed from screen using R commander): They belong respectively to the two ...
0
votes
0answers
142 views

Using results of Canonical Discriminant Analysis to get overall variable importance?

I have a dataset with thousands of observations pre-assigned to 18 groups and with measures for 8 different variables. I am using canonical discriminant analysis to see how separable my 18 groups are. ...
1
vote
0answers
90 views

Discrimination between measurements made at different points in time

I would like to ascertain what variables discriminate best between experimental conditions in a repeated-measures experimental design. I have performed Repeated Measures MANOVA to determine whether ...
0
votes
0answers
29 views

How to preprocessing category data for discriminant analysis

I want to do discriminant analysis for five set observation data (five category) using LDA. From references, most of them suggesting to apply some preprocessing first (i.e. transformation, center, ...
0
votes
0answers
78 views

Multivariate normality in Discriminant Analysis when using dummy variables

I've studied statistics now for almost two years and I'm starting to believe I have missed something very fundamental. I'm doing discriminant analysis where, as I understand it, I can use dummy ...
0
votes
1answer
193 views

LDA, PCA and k-means: how are they related?

I am trying to understand how linear discriminant analysis (LDA) is related to principal component analysis (PCA) and k-means clustering method. As an example, here is a comparison between PCA and ...
0
votes
0answers
29 views

LDA with three variabilities

In a classical linear discriminant analysis (LDA), we usually have only two variabilities to handle (the between classes and the between individual variability). Usually LDA is used to maxmize the ...
1
vote
0answers
20 views

Individual factor significance in multilevel sPLS-DA

I recently was asked by reviewers to "include p-values" with my multilevel sparse partial least squares analysis. In brief, I have a nested design with two factors, say treatment and sampling region. ...
0
votes
0answers
247 views

Explanatory variables with linear discriminant analysis

I have a set of data which contains a mix of continuous, categorical variables etc. I wanted to apply linear discriminant analysis however, from some research on the Internet I understand that ...
3
votes
2answers
156 views

Dimensionality reduction technique similar to LDA when class labels are probabilistic

Given discrete class labels, say True and False, LDA (linear discriminant analysis) can be used to perform discriminant dimensionality reduction and attempt to find a subspace that best separates the ...
2
votes
1answer
698 views

How to follow up a factorial MANOVA with discriminant analysis?

This is a follow-up to to my previous question: How can MANOVA report a significant difference when none of the univariate ANOVAs reaches significance? I have two IVs with each having three levels ...
1
vote
1answer
84 views

When to use linear discriminant function and when logistic regression? [duplicate]

I am trying to find out when for creating classification rule to use linear discriminant function and when to use logistic regression? I need to help to find information sources to this topic. Any ...
2
votes
0answers
37 views

Posterior probabilities as LDA output [closed]

My colleague found @MichaelChernick's post on collinear variables in multiclass lda training. It may be the answer to a very tough question I have had a hard time getting answered. Can someone confirm ...
0
votes
0answers
53 views

Best time to contact: logistic regression loop or discriminant analysis

I want to do a contactability model which gives for each phone the best combination of time-day to be called. My data is the register of each trial and the result (contacted or not) by day and time. ...
1
vote
1answer
166 views

What is the correct formula for covariance matrix in quadratic discriminant analysis (QDA)?

I know that in quadratic discriminant analysis (QDA) we use the variance of each class, so is the formula different than that in linear discriminant analysis (LDA)? Is it $$\frac{1}{N-K} \sum (x - ...
1
vote
1answer
154 views

Testing Logistic Regression Classifier in R

I am testing the logistic regression classifier in R. I created some test data like this: x=runif(10000) y=runif(10000) df=data.frame(x,y,as.factor(x-y>0)) ...
0
votes
0answers
311 views

How do we generate the ROC curve for Linear Discriminant Analysis method

I know the method to generate the ROC curve for other methods such as naive Bayes where the tuning parameter is the threshold like also in logistic regression. If we want to generate the ROC curve ...
1
vote
1answer
19 views

Extract features to explain different states of the world

I have a problem that can be seen as the inverse of a classification problem. I don't need to classify points, but to explain the differences (if any) between points in different, pre-specified ...
3
votes
1answer
951 views

What is the correct formula for between-class scatter matrix in LDA?

At one point in the process of applying linear discriminant analysis (LDA), one has to find the vector $v$ that maximizes the ratio $vBv'/vWv'$, where $B$ is the "between-class scatter" matrix, and ...
2
votes
2answers
80 views

Log transformation for data?

If the data is between (0,1) because of some kind of vector normalization to get rid of background noise, is it still OK to do log transformation to improve normality? Or we have to do logit ...
0
votes
1answer
51 views

Cannot perform tests for multivariate normality. Is my data set too large?

I'm examining the performance of quadratic and linear discriminant models at classification. My dataset has 250,000 observations, 2 groups and 30 explanatory variables. I thought it would be worth ...
4
votes
1answer
190 views

How can MANOVA report a non-significant p-value while LDA results in perfect separation of two groups?

I am new to statistics and currently got a dataset which contains $80$ dependent variables and $1$ independent variable with $2$ groups. MANOVA reports a $p$-value of $> 0.6$ on this dataset. But ...
0
votes
0answers
19 views

linear discriminant analysis, Bayes approach authors?

I know that in 1936 Fisher proposed the LDA that minimizes the variance within and maximizes between. My question is, the Bayes approach of LDA is attributed to a particular(s) author(s)? and what ...
1
vote
0answers
17 views

Canonical discriminant analysis - lack of equality of covariance matrices [duplicate]

I have a dataset with 92 observations and two groups that corresponde to two analytical fractions of soil samples (i.e., light fraction or LF, and mineral-associated fraction or MoM). Each group has ...
1
vote
0answers
54 views

What are F values of the canonical functions in discriminant analysis?

Recently, I read a paper (Pell et al. 2009) in which the authors use Discriminant Analysis, and I quote: The discriminant analysis produced three significant canonical functions (Function 1, ...
0
votes
1answer
560 views

More recognizable Python implementation of Linear Discriminant Analysis?

I have been using scikit-learn's LDA implementation to do some experiments, and recently wanted to test out some modifications to the LDA derivation. I was looking at the Python implementation that ...
0
votes
0answers
110 views

What is a “discriminant function” and how to interpret it?

I want to test a model using discriminant function analysis. My question, as the title states, is very basic: What is a discriminant function? That is, how can I interpret the different discriminant ...