I am working on the following regression problem:
- 1 dependent variable
- m continuous features
- 1 categorical feature c with n possible values (giving a total of m + n continuous features)
My initial exploration of the data suggests that different subsets of the data (characterized by the value of c) favor different features in terms of correlation with the dependent variable. Currently, my best model is a linear regression model using only one of the available features. Multiple linear regression did not improve the model because of the aforementioned diversity in the underlying data.
However, I was wondering if it would make sense to fit n different models for each value of c instead of using one model for all possible values of c. Implementing this approach in python using scikit-learn is straightforward. I am however left with three questions:
- Are there any mathematical drawbacks with this idea?
- How do I compare the resulting n models to my other model? I am unsure on how to interpret the n resulting metrics (i.e., a cross-validated r squared score) to those of my other model.
- Is there a python/sklearn implementation of my idea? Right now I manually split the input depending on the value of c when training and evaluating the models.