This is the first time I am posting a question, so please excuse any etiquette violations and poorly worded questions!
I am working on the analysis for a chapter of my thesis. I am examining the behavioural response of an animal to a visual stimulus, and trying to determine which of eight explanatory variables (and their two-way interactions) affect this response. I recorded the response on an ordinal scale of 0 (no response), 1 (attention to but no avoidance of stimulus) or 2 (escape response to stimulus). I am leaning towards collapsing categories and using logistic regression where a 1 is an escape response, and 0 is anything else because logistic regression seems much easier to interpret.
I have 794 observations. I am including observer and location (because field sites differed) as random effects, although I am unsure this is a good approach.
I am having trouble with model selection. I ran all possible subsets using the dredge function in packing 'MuMIn'. I thought I was avoiding data dredging by
- including main effects which were selected because I thought they would have an effect (rather than all conceivable variables)
- including only the two-way interactions of interest (R will not run if the global model includes all possible two-way interactions because of the huge number of terms/models)
I've come to realise that the second point may be problematic because it leads to an unbalanced model set as in Burnham and Anderson (2002).
Page 169: When assessing the relative importance of variables using sums of the AIC weights, it is important to achieve a balance in the number of models that contain each variable j.
My questions are
Is it possible to have a balanced model set without it being considered data dredging? If so, how?
Is my approach at all reasonable? If not, are there other avenues I should explore? I started with Hosmer&Lemeshow purposeful forward selection, as advocated by my supervisor, but I had some issues with this which I can elaborate on if necessary.