In what cases linear SVM and LDA are equivalent? Lately I've been training some binary classifiers for a school project, and I realized that my linear SVM in most cases gives almost identical AUC scores as my LDA classifier.
I used C=1 for the SVM and automatic shrinkage (using the Ledoit-Wolf lemma) for the LDA.
What circumstances can produce such similar results for two classifier having so little in common?
 A: When you have very strong separation between classes based on your (obviously) strong predictors, many algorithms will end up with similar performance. You may get the same accuracy and AUC using decision trees or forests, neural networks, etc. That this occurs despite vastly different modeling architectures is driven by the separability of the data. 
A: Since more than 20 years, classifiers are being combined into 'bags' (total classifiers), using summing or voting schemes. It is well-known that very different classifier algorithms can be combined into really well-performing 'total classifiers'. Such a total classifier commonly outperforms all the individual classifiers that are part of it.
This is clear evidence that classifiers perform differently, especially on difficult classification tasks. Classifier choice does matter.
Reference:
J. Kittler, M. Hatef, R.P.W. Duin, J. Matas. "On combining classifiers," IEEE Transactions on Pattern Analysis and Machine Intelligence, Volume 20, Issue 3, pp. 226 - 239, 1998.
