In the context of classification on somewhat large datasets (say at least 50Kx50K), I am wondering in which cases non-linear models are superior to linear ones to warrant the added complexity. I often see in my own research that for these larger datasets, non-linear datasets cannot outperform linear ones (say for a linear kernel SVM and an RBF kernel SVM). But this might be biased due to my 'repository selection' of datasets which are all sparse and drawn from transactional data.
My intuition says that specifically for an RBF kernel, a linear kernel should be a lower-bound for the performance that you can achieve with an RBF kernel, but my hope of attaining higher performance than this lower bound is not fullfilled because in the end they all achieve more or less the same performance.
Specifically my question is this: have you encountered situations in which non-linear models were worth the effort? Or do you perhaps know about research confirming/rejecting my own observations?