To include or not include a predictor: making a model I am in an animal ethology study and I want to make a linear model while observing certain animal behaviours. 
Questions: 
1. Is there a general rule on what we can include or not as predictors? (trying to avoid the stepwise regression problem)
2. How will be know that 'enough' is enough? Adding too much predictors (like shooting arrows blindly), or adding too few (too inflexible, or biased?)
 A: When modelling something from 'real life', I've always understood there's roughly two goals in the field of Epidemiology (this is probably not completely comprehensive for all fields, but helps me explain): 


*

*Explaining reality / finding causal factors for some outcome

*Predicting (future) events/values without measuring these outcomes themselves
In medical research the first is called causal research. If you are making these kinds of models, you usually have one factor of interest (the 'determinant'), and want to examine if and how this determinant influences a certain outcome. The goals of modelling in this kind of research is to eliminate the effects of other factors, so that any effect measured can be attributed to the factor of interest. These other factors are then called confounders. 
If your research is set up as a causal study, my suggestion would be:
Selection of which confounders to add is recommended to be done based on biological reasoning. The leading question whether some factor is a possible confounder is whether it can influence the level of/occurrence of the determinant AND the level of/occurrence of the outcome. Note that this is not an issue which statistics should answer. Often, the methods of these studies is to think of confounders in advance, and model without feature selection techniques.
The second is called diagnostic or prognostic research. When making diagnostic or prognostic models in the biomedical field the goal is to predict the level of the outcome, of the probability of its occurrence. Important to realize is that these types of research do not (usually) have a single factor of interest. Instead, all available information can be considered! However, when making such models it is paramount to try to keep the final models as simple as possible, otherwise future users need to collect heaps of data, where some variables only provide meaningless improvements of predictive accuracy. That's why we apply feature selection. As far as I know, there's only rules of thumb here:


*

*focus on predictors with some plausible relation with outcome based on prior research as they are most likely to have any predictive value.

*focus on predictors which are easy to measure/obtain for future users of the model, and only use difficult to obtain predictors when they are expected to improve predictive performance.

*include 'the usual suspects' (for medical research, stuff like age, sex, smoking and body mass index are often included because they are surrogates for a lot of difficult to measure factors; e.g. age often is a good surrogate for physical and mental frailty)


See the TRIPOD statement for more help on these latter type of research.
Now in both the first and the second type of research, the amount of predictors to be taken into account when using modelling techniques is an area of discussion. 'Statistically', a model's performance (or fit to the data) will (almost) always increase when fitting more predictors to a regular (linear) regression model. However, due to the increasing parameter spaces the statistical model needs to extrapolate to, the uncertainty of the estimated values grows and there is a limit to the amount of predictors which can reasonably be fit. In causal research however, leaving out confounders will mean you cannot fully attribute the effects found to your factor of interest (and this will mean failing to answer the research question). That's why the study design of such studies include 'non-statistical' techniques to avoid confounding altogether (think of randomization in trials), or to minimize the effects of the major confounders (matching, but this technique is heavily discussed).
For predictive research, missing certain factors is not that much of a problem. (as stated above you might even keep some out because they are too difficult to obtain). Instead, the statistical procedures dictate the amount of features you can study. A general rule is to test one predictor for every 10 outcome events (of the least frequent outcome category, if the outcome is categorical) or for every 10 units of data (it the outcome is continuous). This rule is heavily debated though. And note: this rule of thumb is meant to be used to calculate the amount of possible predictors before feature selection techniques are applied! Finally, the most important for this type of research is the way you select predictors as certain techniques influence the external validity/out-of-sample performance. Like you said, you want to avoid stepwise regression. LASSO or elastic net selection techniques are what is now recommended.

To conclude, although there is no specific rule which can you tell exactly which and how many predictors to select, there are some starting points based on sound reasoning outside of your data. For causal studies, focus on confounders and include as much as the data will allow, while refraining from model selection techniques.For predictive research, use the rule of thumb to find out how many predictors can reasonably be tested and then start with selecting proven/assumed (biological) causal factors, add the usual suspects, while keeping in mind the ease-of-use of all these predictors.  
A: To compare linear models one can use an F-test. An F-test helps to assess if error variance is reduced if an extra feature is added in an extended model (or vice versa removed).
The relevant quantities used are the error variances of the respective models and the degrees of freedom of the respective models.
An example can be found here.
