I'm looking at Fisher's LDA on various datasets on UCI ML repository and trying to see where LDA might perform badly. One reason I can think of is if the data distribution is not a multi-variate normal distribution. This is from the fact I read in a book where you apply LDA on multivariate normal distribution. Is that thought process correct? When might LDA give bad results?
By definition the random vector $X$ is multivariate normal if all linear combinations $a^T X$ have some (univariate) normal distribution. So one idea to test multivariate normality is to search among the vectors $a$ for one such that $a^T X$ is definitely not normal. That is the idea behind pp, projection pursuit methods. See https://en.wikipedia.org/wiki/Projection_pursuit
A fast way of examining whether your data set is Gaussian distributed or not is to plot a histogram for each variable of your data set (if the dimensionality is small), or simply just calculate the sample skewness and kurtosis to check if they are Gaussian distributed. A Gaussian distributed data set will have skewness = 0 and kurtosis =3.