I want to understand how some dependent variable
y, depends on a known relationship with independent variable
x, but also how
x potentially interacts with a high dimensional complex set of features (microbiome data which can be represented as thousands of predictors per observation of
y). Therefore, I know that the general form of the model is:
y ~ m1*x + b
However I would like to add an interaction term, generating the model:
y ~ m1*x + m2*x*microbiome + b
microbiome is the very large microbiome feature set.
I think the microbiome data essentially interacts with the independent variable
x to affect
y, but I don't know how. There are thousands of species, and my guess is that certain combinations will be predictive of this interaction and explain a lot of variation, but a priori I don't know which. I also suspect that the microbiome features relate to each other in a non-linear way. Essentially I want to use a machine learning approach to figure that out. I am aware of "boosted" regressions, where you use a machine learning algorithm on the residuals, however I want to specify something a bit more mechanistic than that.
If anyone could suggest a method to do something like this (especially if it can be implemented in R) I would be very interested. If there is a way to use the residuals of the model to do this, I would also be interested. It's worth noting that in the actual application I have many more predictors in the model, many of which have non-linear relationships with