Qualitative implications of Linear Discriminant Analysis (LDA) I'm a beginner to LDA, and my question is about its qualitative implications. Say I have two classes of medical data, already classified as:
$C_1=\{x_{11},x_{12}, ...x_{1n_1}\}, C_2=\{x_{21}, x_{22} ...x_{2n_2}\}$. Let's assume that each data lies in $\mathbb{R}^m$, corresponding to $m-$ number of features from a patient, say height, weight, arm-length etc. Assume that the patients in  $C_1$ have a certain disease, and the ones in $C_2$ do not. No distributional assumptions are given, and we also don't assume that the covariance matrices for these two classes are the same. (Apprantly, in this note http://www.di.univr.it/documenti/OccorrenzaIns/matdid/matdid437773.pdf that I just studied, they didn't have this assumption, although the wiki page mentions it.) But one can use central limit theorems if necessary. Now, we use linear discriminant analysis (LDA) and find a line or direction $w$ that maximizes the class separation between $C_1, C_2.$
What will that imply from the view point of the disease, or medically?
 A: I previously worked on classification of biomedical data.
The implication of a LDA model is that there's a discriminant hyperplane given by the linear combination of your features which, given the projection of your data on this subspace, maximizes the interclass variance and minimizes the intraclass variance. It's dimension is $\min\left(N,C-1\right)$, $N$ is the feature space dimension and $C$ is the number of classes. And that's it, no explicit medical implications.
Or, in other words, the class centroid will be the furthest apart they can be while the instances from the same class will be the nearest possible given the LDA conditions.
It's of your best interest to have the features be dimensionless, and most probably normalized aswell. Also, in your case, the hyperplane is actually a single dimension. 
Now, it could be used to infer a rule to classify new unlabelled cases, if you can show that it's reliable and accurate given your goals. It's a single value given by the linear combination of other values, and the possibility of a parametric solution for classification makes it interesting.
LDA itself doesn't give you such a rule, as it's only a dimensionality reduction method. It gives you the weights to the discriminating subspace, where you could apply any rule to the classification. A common approach is to define a rule based on the distance to the centroids of the classes in this hyperplane, but any other rule can be applied.
In this case, the projection is given by (Derivations in this lecture).
$$Y = W^{T}X$$
The weights $W$ are found through the maximization of the Fisher Criterion $J\left(W\right)$
$$W^{T}=\underset{W}{\operatorname{argmax}} \{J(W)\}=\underset{W}{\operatorname{argmax}} \left\{\frac{W^{T}S_{B}W}{W^{T}S_{W}W}\right\}$$
See $J(W)$ depends of the between classes scatter $S_{B}$ and the within classes scatter $S_{W}$. If you add one more dimension, a new feature, to the problem, $S_{W}$ necessarily increases. The only way to make your new $J(W)$ bigger than the other is if your new $S_{B}$ increases the criterion just as much $S_{W}$ diminished it, or in other words, if you obtain a bigger $J(W)$ with more features it's because said features increase the class separation.
If the rule you created in the discriminant subspace is as accurate as the medical diagnosis, congratulations. If it isn't it can still be (probably is) useful, as it's a summary of all the information, and that would allow you to develop, say, a Decision Support System.
