As AdamO suggests in the above comment, you can't really do better than read Chapter 4 of The Elements of Statistical Learning (which I will call HTF) which compares LDA with other linear classification methods, giving many examples, and also discusses the use of LDA as a dimension-reduction technique in the vein of PCA which, as ttnphns points out, is rather popular.
From the point of view of classification, I think the key difference is this. Imagine that you have two classes and you want to separate them. Each class has a probability density function. The best possible situation would be if you knew these density functions, because then you could predict which class a point would belong to by evaluating the class-specific densities at that point.
Some kinds of classifier operate by finding an approximation to the density functions of the classes. LDA is one of these; it makes the assumption that the densities are multivariate normal with the same covariance matrix. This is a strong assumption, but if it is approximately correct, you get a good classifier. Many other classifiers also take this kind of approach, but try to be more flexible than assuming normality. For example, see page 108 of HTF.
On the other hand, on page 210, HTF warn:
If classification is the ultimate goal, then learning the separate
class densities well may be unnecessary, and can in fact be
misleading.
Another approach is simply to look for a boundary between the two classes, which is what the perceptron does. A more sophisticated version of this is the support vector machine. These methods can also be combined with adding features to the data using a technique called kernelization. This does not work with LDA because it does not preserve normality, but it is no problem for a classifier which is just looking for a separating hyperplane.
The difference between LDA and a classifier which looks for a separating hyperplane is like the difference between a t-test and some nonparamteric alternative in ordinary statistics. The latter is more robust (to outliers, for example) but the former is optimal if its assumptions are satisfied.
One more remark: it might be worth mentioning that some people might have cultural reasons for using methods like LDA or logistic regression, which may obligingly spew out ANOVA tables, hypothesis tests, and reassuring things like that. LDA was invented by Fisher; the perceptron was originally a model for a human or animal neuron and had no connection with statistics. It also works the other way; some people might prefer methods like support vector machines because they have the kind of cutting-edge hipster-cred which twentieth-century methods just can't match. It doesn't mean that they're better. (A good example of this is discussed in Machine Learning for Hackers, if I recall correctly.)