I've been trying to learn more about online learning lately (it's absolutely fascinating!), and one theme that I haven't been able to get a good grasp on is how to think about model selection in offline versus online contexts. Specifically, suppose we train a classifier $S$ offline, based on some fixed data set $D$. We estimate its performance characteristics via cross-validation, say, and we select the best classifier this way.
This is what I've been thinking about: how, then, do we go about about applying $S$ to an online setting? Can we assume that the best $S$ found offline will also perform well as an online classifier? Does it make sense to gather some data to train $S$, then take that same classifier $S$ and "operationalize" it in an online setting with he same parameters found on $D$, or might another approach be better? What are the caveats in these cases? What are the key results here? And so forth.
Anyway, now that's out there, I guess what I'm looking for is some references or resources that will help me (and hopefully others, who have been thinking about this kinda thing!) make the transition from thinking solely in offline terms, and develop the mental framework to think about the issue of model selection and these questions in a more coherent way as my reading progresses.