# LDA cutoff (decision boundary) value

I have run a linear discriminant analysis for the simple 2 categorical group case using the MASS package lda() function in R. With priors fixed at 0.5 and unequal n for the response variable of each group, the output basically provides the group means and the LD1 (first linear discriminant coefficient) value. There is no automatic output of the cutoff (decision boundary) value estimated that is later used to classify new values of the response variable into the different groups. I have tried various unsuccessful approaches to extract this value. It is obvious that in the simple 2 group case the value will be close to the mean of the 2 group means and that the LD1 value is involved (perhaps grand mean * LD1?). I am probably missing (misunderstanding?) the obvious and would appreciate being educated in this matter. Thanks. Regards,BJ

-
Please explain what is "LD1 value" in MASS output –  ttnphns Sep 23 '11 at 21:35
LD1 is the first linear discriminant coefficient –  BJessop Sep 24 '11 at 0:14
If the OP could explain precisely what LD1 is, there would probably not be a question. As I see it, you are not supposed to use the components of an lda-object from an lda-call directly, but rather use the plot or the predict methods implemented for these objects. That does not mean that it is not interesting to figure out what the components are, and I have tried to give some explanation and further references in my answer. –  NRH Sep 24 '11 at 7:54

The LD-vectors, which are in the scaling entry of the list returned by ldain R, are not so easily comprehended. The help page says that scaling is

      a matrix which transforms observations to discriminant
functions, normalized so that within groups covariance matrix
is spherical.


which is not super informative. Inner products with these vectors give the coefficients for a projection to a $K-1$ dimensional space (with $K$ groups, that is, a one-dimensional space if there are two groups). Everything is centered using the total mean of the predictors before projection. In this projection, classification happens to the group with the nearest mean, as measured by the usual euclidean distance, if the prior probabilities are equal. Perhaps the best thing to do to understand precisely how the computation of the predictions work is to read the R-code in MASS:::predict.lda. The MASS book by Ripley and Venables may also be useful, but I find that Chapter 3 in Ripley's other book Pattern Recognition and Neural Networks is more complete. Here you will find the complete explanation behind the computations implemented in the R function lda, and a more detailed description of what the projection actually is.

For the two group case things simplify, the LD1 vector is then proportional to $\hat{\Sigma}^{-1}(\hat{\mu}_2 - \hat{\mu}_1)$, and the linear discriminator is given by taking inner product with LD1. In the case of equal prior probabilities on the two groups you don't need to know the constant of proportionality and the cutoff is $\frac{1}{2}(\hat{\mu}_2 + \hat{\mu}_1)^T\text{LD1}$. With unequal priors it seems to me that you need to know the constant of proportionality to compute the cutoff in this formulation. In the alternative, and this is essentially what MASS:::predict.lda does, you can for a given $x$ compute the inner products $c = x^T \text{LD1}$ and $c_i = \hat{\mu}_i^T \text{LD1}$ for $i=1,2$ and then $$d_i = \frac{1}{2}(c - c_i)^2 - \log \pi_i.$$ Finally, you classify to the group with the smallest $d_i$.

-
Thanks for the references. In the simple 2 categorical group case that I am concerned with I had hoped that the lda function in R would allow easy extraction of the discriminant function threshold value that is calculated and used in the classification process applied to new data. Apparently it is not directly available. I have done the discrimination and classification steps on my data. I need the decision boundary value (in this simple case, a single value) for another purpose and general interest. In this simple case, if lda() does not provide it, how can I calculate it by hand? –  BJessop Sep 24 '11 at 13:47
@BJessop, I have tried to provide a more detailed explanation for the two group case. –  NRH Sep 26 '11 at 10:14