As the title suggests, I am wondering why we consider LDA a linear classifier.
More specifically, I would like to know why LDA is considered a linear classifier in the following case:
- The response consists of two classes, coded as 1 and 0.
- The threshold for a given observation to be classified as 1 is $a$, where $a \in [0,\,1]$.
A simpler problem
The second condition is very important -- I know why we consider LDA a linear classifier in the case where we classify an observation as "1" in the case we estimate the posterior probability of this class to be greater than 0.5.
Namely, one proof could involve the use of the disciminant functions corresponding to each of the two classes. Since the ranking of the discriminant functions is always equal to the ranking of the posterior probabilities, $p_1(X) = p_0(X) \iff \delta_1(X) = \delta_0(X)$. Since this is the decision boundary, and $\delta_1(X) = \delta_0(X)$ describes a hyperplane in $X$, we consider the model linear.
In this more general problem, I can no longer use $\delta_1(X) = \delta_0(X)$ to derive the decision boundary. Instead, the decision boundary is of the form $p_1(X) = a$. I don't think it is particularly feasible to show that this equation is linear in $X$, and so I wonder if it is possible to express the equation in an equivalent form $\delta_1 = b$, where $b$ is independent or linear in $X$. Then the conclusion that LDA is linear in this case would follow. It is just that I cannot think of how to derive this $b$, either.