I am attempting to compare behavioral responses across two species (one native and one invasive). Predictors run the range of types including continuous (size), discrete (day of trial) and categorical (all my treatments, for example, competitor ID; conspecific or heterospecific). Responses are in the form of proportion of time performing behavior XX, with 5 different behaviors being observed. Because my RV's include values at zero (approximately 70% of the observations) and sometimes one, I have been using the GAMLSS package in R to run zero-and-one beta regression. Initially, I did this across the 2 species independently, as the native species is the primary focus in my research and I simply sought to catch effects of the non-native in the competitor parameter. However, it came to be apparent that in order to explain the behavior of one, it became necessary to explore both. And this brings me to my question. I am seeking ways to compare responses across the 2 species. In most cases, the responses are of a similar nature, potentially only varying in magnitude (ie. both species should hide when at risk, but one is more bold than the other). Although this is not always true, particularly at the nu-level of the models.
I have been working with 2 possible options. 1) Run the a model with all data and simply include a species parameter and compare effects through the Species:XXX interactions. This is troublesome in the number of interactions, and the fact that not all treatments seem to be significant across the two species. As well, one of my treatments holds 3 levels. It seems there's no way to obtain an effects estimate on the parameter as a whole (as in an anova or deviance table) with a GAMLSS object of my type. 2) Calculate relative variable importance. The issue here is that I know of no other way in GAMLSS to perform variable selection than stepwise, and I realize that's become very taboo. But as I read here, it seems as though running all the combinations of parameters is no better. Also, I'm aware of the necessity of balanced numbers of model sets across parameters (including the multiple interaction terms) and that the interactions must be compared apart from primary terms. Using my knowledge of the biology to develop a set of a priori models is a possibility. Unfortunately, I cannot really knock out the interactions as they are biologically relevant (and often where the significance lies). Also, the delta AIC's are often quite close leaving a very large number of candidate models and questioning the validity of the model selection process itself.
Is there a defensible way to get this comparison across my species, either through one of these methods or another that I haven't considered? In the past, people would have simply have run a bad transformation and hoped for the best with a linear model. Even going down the zero-and-one beta route puts me into waters not many go with ecological data. The less than a handful of papers published with this analysis method do not even bother to get this far into the data and so are little help (in other words, they do stepwise selection and just report the "best" model). I've read the other thread here regarding the use of LASSO in these types of situations with model selection. However, I have to say I'm really beyond my ability with stats now outside of package solutions. Ultimately, the objective has to be to compare the magnitude of the responses across my two species in relation to the predictors.
Sorry if any of this is confusingly worded. I'm just treading water with this at this point and I've exhausted resources locally. Thanks.