5
$\begingroup$

I'm looking at Fisher's LDA on various datasets on UCI ML repository and trying to see where LDA might perform badly. One reason I can think of is if the data distribution is not a multi-variate normal distribution. This is from the fact I read in a book where you apply LDA on multivariate normal distribution. Is that thought process correct? When might LDA give bad results?

$\endgroup$
  • 1
    $\begingroup$ I don't quite follow the phrasing of your question. Fisher's LDA assume the data are multivariate normal, which would be violated if there were, eg, some binary variables as well. $\endgroup$ – gung - Reinstate Monica Oct 28 '12 at 3:36
  • 2
    $\begingroup$ I think you are missing a "not" before "a multi-variate normal" distribution, so I added that. Feel free to change it back if I've messed things up. $\endgroup$ – Peter Flom - Reinstate Monica Oct 28 '12 at 12:21
  • 1
    $\begingroup$ If you Google "LDA and logistic regression" you will find a lot of papers on this e.g. Pohar et al $\endgroup$ – Peter Flom - Reinstate Monica Oct 28 '12 at 12:27
  • 1
    $\begingroup$ @gung I'm looking at ways to see if before applying LDA can I do something to see if data is in the form of multi-variate normal distribution. I might be getting my phrasing incorrect. $\endgroup$ – gizgok Oct 28 '12 at 20:28
1
$\begingroup$

By definition the random vector $X$ is multivariate normal if all linear combinations $a^T X$ have some (univariate) normal distribution. So one idea to test multivariate normality is to search among the vectors $a$ for one such that $a^T X$ is definitely not normal. That is the idea behind pp, projection pursuit methods. See https://en.wikipedia.org/wiki/Projection_pursuit

$\endgroup$
0
$\begingroup$

A fast way of examining whether your data set is Gaussian distributed or not is to plot a histogram for each variable of your data set (if the dimensionality is small), or simply just calculate the sample skewness and kurtosis to check if they are Gaussian distributed. A Gaussian distributed data set will have skewness = 0 and kurtosis =3.

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.