Say you have sets $S_1, \dots, S_n$, where each $S_i$ a set of possible features to use. Rather than searching blindly (as in a typical feature selection problem), you've chosen these sets using some kind of prior knowledge. This gives models $M_1, \dots, M_n$, where each will be trained using the corresponding set of features. The problem is to select the model with best generalization performance. This is a standard model selection problem, and can be tackled as such.
For example, use k-fold cross validation. The training set is used to train the parameters of each model, and the validation set is used to select the best model (and any hyperparameters). Keep in mind that it's possible to overfit the validation set if you compare too many models (or hyperparameter choices) relative to the amount of data on hand (see Cawley and Talbot 2010). One can easily run into problems by blindly searching for features this way. But, if you're just comparing several models that you've carefully chosen using prior knowledge, you should be ok.
Because the validation set has been used for model selection, validation set error should not be used to estimate generalization performance of the final model, as it would be optimistically biased (see here). Instead, estimate generalization performance of the final model using an independent test set or, alternatively, use an outer cross validation loop (i.e. nested cross validation).
Cawley and Talbot (2010). On Over-fitting in Model Selection and Subsequent Selection Bias in Performance Evaluation.
