When do linear classifiers work better than non-linear classifiers? I am currently working on a classification problem, and I am observing that the linear classifiers are outperforming the non-linear classifiers. This is very unintuitive to me. What could be the cause of this?


*

*training data? 

*number of features? feature types?


I hypothesize its the lack of training data that is the cause of this, but my google-fu hasn't been good enough to find any scientific articles suggesting so.
 A: If you are truly interested in all-or-nothing classification you can fool the proportion classified correctly with a variety of bogus models.  If you are interested in prediction instead, and use proper accuracy scoring rules, you'll see that what outperforms other methods in a variety of situations is additive models that allow predictors to act nonlinearly (e.g., regression splines).  You could call this class of models generalized additive models (GAMs, which is strictly speaking for the nonparametric case) or additive smooth models.
The reason that additive models, or linear models for that case, can outperform other methods in many situations is that they are effectively Bayesian with a prior distribution that places weight on additive effects and places little weight on non-additive (interactive; synergistic) effects.  Use of prior information can really boost mean squared error (and other measures) of predictive accuracy.  We find in many situations that the dominant effects are additive, and complex interactive effects (of the type featured by random forests, SVMs, recursive partitioning, and other approaches) are not very predictive.
A: If a problem is nonlinear and its class boundaries cannot be approximated well with linear hyperplanes, then nonlinear classifiers are often more accurate than linear classifiers. If a problem is linear, it is best to use a simpler linear classifier.
The performance depends on the problem.
A: What sort of nonlinear classifier you're using matters. Supposing that there is an underlying linear relationship, a polynomial fit will obviously work at least as well as a linear fit but a decision tree that uses a sequence of binary divisions will require an arbitrary number of divisions to capture the underlying relationship.
A: "Performance" is a loaded term.
Variations on meaning:    


*

*How long it takes to compute (seconds, cycles, ...)

*How much memory it uses during compute (on-die Bytes, L2, RAM, cache to disk)

*How big is your favorite fit metric ($R^2$, AIC, ROC, ...)
I suspect that:
It is really a linear problem, then the nonlinear can model part of the noise with the signal, and a decent fit metric can indicate that.  Nonlinear fits also take longer to compute than linear fits. 
When you say "performance", what exactly do you mean?
