Regressors vs. conditioning variables in glmtree I have a dataset with ~800K samples, ~300 features and I'm trying to predict a binary outcome. I've started with sklearn's SGDClassifier (using log loss and l1 penalty), and I got a nice 0.67 auc score on my validation set; now I'm trying to improve this.
I have a reason to believe that different subspaces of features space "behave" differently, so I thought fitting a glmtree would make sense (splitting the features space to different subspaces and fitting a logistic regression in each of them). However, when trying to write the formula, I don't have any prior knowledge regarding which features should be used as predictors for the glm, and which should be used as conditioning variables. 
Is there any automated way to select which is which?
 A: glmtree() is particularly strong in situations where you have a GLM that you would typically fit to the whole sample, e.g., a voter targeting model (Rusch et al. 2013), a treatment effect model (Seibold et al. 2016), or an economic growth model (Wagner & Zeileis 2017). Then you can detect heterogeneity in the model parameters depending on the partitioning variables.
If you have no such "base" model and have no idea which variables are important, then you can fit an intercept-only GLM in each node. However, then the resulting tree is often rather similar to other classification trees (CART, CTree, etc.).
I have also seen situations where all numeric variables have been used in the regression and the categorical variables in the partitioning part. Possibly the numeric variables can also appear in both parts. However, this is only likely to yield good results if the number of numeric variables is small to moderate.
In your case with ~800K observations and ~300 variables and no prior information I personally would not bother with single trees anyway. I would probably use a random forest. You have enough observations to approximate potentially linear (or partially linear) effects sufficiently well through the forest.
References


*

*Thomas Rusch, Ilro Lee, Kurt Hornik, Wolfgang Jank, Achim Zeileis (2013). Influencing Elections with Statistics: Targeting Voters with Logistic Regression Trees. The Annals of Applied Statistics, 7(3), 1612–1639. doi:10.1214/13-AOAS648

*Heidi Seibold, Achim Zeileis, Torsten Hothorn (2016). Model-Based Recursive Partitioning for Subgroup Analyses. The International Journal of Biostatistics, 12(1), 45–63. doi:10.1515/ijb-2015-0032

*Martin Wagner, Achim Zeileis (2017). Heterogeneity and Spatial Dependence of Regional Growth in the EU: A Recursive Partitioning Approach. German Economic Review. Forthcoming. doi:10.1111/geer.12146
