Can any "model" be cross validated? Suppose you have a dataset that has heights of people and whether they play basketball professionally. You have a theory that anyone taller than 200 cm plays professional basketball and everyone shorter does not. This is your "decision rule" (i.e. model).
If you took 20% of this data and tested this theory - measured how well this decision rule predicted if someone plays professional basketball: is this a statistically valid approach to test how well this rule might "generalize" to unseen data? Then repeat this several times on random 20% subsets of the data?
Does this sound reasonable?
 A: The point of having a held-out test set is that your machine learning algorithm learned the rules from the data, so you cannot validate them on the same dataset, because you’d risk overfitting. If you derive your rule using exploratory data analysis or some kind of algorithm then the proper way would be to derive the rule using only training data and validate it using a test set. On another hand, if you made the rule a priori of looking at the data, just test it on the full dataset.
Moreover, your rule sounds like a hypothesis to be tested, if yes, we go into the realm of statistics and hypothesis-testing, rather than building predictive models. If that’s the case, I’d recommend you consult a statistics handbook beforehand, because statistics often use their methodologies, different from machine learning. For example, if you tested multiple rules like this (“let’s try 20%”, “ok, now 22%”, …), this is a multiple testing problem, and you risk making false discoveries. You would “overfit”, but cross-validation would not help here.
A: Your example is not a model, but a decision rule. A model would predict a probability
$$P(basketball\mid height)$$
Modeling this probability could be done parametrically (e.g. logistic regression) or non-parametrically (e.g. kNN, or kernel density estimation as the R function cdplot does). A decision rule is then derived from the model on basis of a trianing set, and this decision rule can be evaluated by cross-validation.
That textbooks discussing classical statistical models (e.g. logistic regression,, which is a special case of "generalized linear models") rarely suggest cross-validation for model validation might be a historical legacy: The theory of linear models predates the wide-spread use of cross-validation. Actually, all textbooks that I have read so far about linear models only suggest a resubstitution of the training set into the model for evaluation. This does not mean, howver, that cross-validation cannot be used and it should provide a less biased method for evaluating a model.
A: Yes every model can be cross validated. That is, every scientific model can be cross validated.
This relates to the theory of science of Popper about falsification. Scientific theories/models should be able to provide predictions according to which they can be tested.
From this point of view 'cross validation' is just the testing of previously established theories/models.
In the point of view/framework of statistics the 'cross validation' is a more elaborate scheme of testing different models based on the data. In a first step the unknown parameters in the theories are determined based on data (which might lead to over-fitting and might lead to the wrong idea that the theory/model works well), after that an additional cross validation step on new data is used to verify and select the best working models/theories.

So there are two requirements for models to be cross validated.

*

*A requirement that models can be validated As long as the models provide testable results there can be cross-validation.

*A requirement that models need to be validated Cross validation in statistics is in particular used for the cases when models/theories contain unknown parameters that are fitted with data. This fitting makes them prone to over-fitting and cross validation is used to filter/verify/select which model/theory is best.
If there is no fitting then there is neither much use for cross validation.



is this a statistically valid approach to test how well this rule might "generalize" to unseen data?

If you mean 'does this generalize the model' in other regions, ie extrapolation, then it is not true. It only generalizes to new data from the same region (but with different noise).
Cross validation does not allow one to extrapolate. It is only working in the range from which the data is sampled.
The reason to use cross validation, and the idea about 'generalizing to unseen data' is relating to the sample being a specific case/dataset which involves different noise that won't be the same in other data sets.
So we can test/cross-validate models on different data sets with different noise, but this is not the same as allowing to extrapolate the model to other regions, instead it is allowing to "extrapolate" or generalize to other datasets (with other noise) within the same region.
The extrapolation of the model involves extrapolation of the determistic term, and that is not tested with cross validation.
