4
$\begingroup$

I am working on the following regression problem:

  • 1 dependent variable
  • m continuous features
  • 1 categorical feature c with n possible values (giving a total of m + n continuous features)

My initial exploration of the data suggests that different subsets of the data (characterized by the value of c) favor different features in terms of correlation with the dependent variable. Currently, my best model is a linear regression model using only one of the available features. Multiple linear regression did not improve the model because of the aforementioned diversity in the underlying data.

However, I was wondering if it would make sense to fit n different models for each value of c instead of using one model for all possible values of c. Implementing this approach in python using scikit-learn is straightforward. I am however left with three questions:

  1. Are there any mathematical drawbacks with this idea?
  2. How do I compare the resulting n models to my other model? I am unsure on how to interpret the n resulting metrics (i.e., a cross-validated r squared score) to those of my other model.
  3. Is there a python/sklearn implementation of my idea? Right now I manually split the input depending on the value of c when training and evaluating the models.
Improve this question
$\endgroup$
2
  • 2
    $\begingroup$ Going this route is going to worsen your problem, not fix it, because it uses far more parameters to fit the data: it's tantamount to introducing all interactions with the variable c as well as allowing the variance of the response to change with the levels of c. $\endgroup$
    – whuber ♦
    Aug 27, 2020 at 14:16
  • 2
    $\begingroup$ Adding to whuber's comment, you can evaluate the "goodness" of your "separate regressions" approach by fitting all interactions of your $m$ continuous features with your categorical variable (which gives identical regressions as the subsets approach, which you can check for yourself), and then comparing the fit with the model with none of these interactions. If you are in predictive modelling mode, you can check various interaction in/out models using cross-validated root mean square error or something similar. In any case, it indeed makes more sense to do this all in one model. $\endgroup$
    – BigBendRegion
    Aug 27, 2020 at 14:25

1 Answer 1

Reset to default
4
$\begingroup$
  1. Are there any mathematical drawbacks with this idea?

Yes, you will lose statistical power as well as running into multiple testing problems. Don't split the dataset.

How do I compare the resulting n models to my other model? I am unsure on how to interpret the n resulting metrics (i.e., a cross-validated r squared score) to those of my other model.

You won't have to if you don't split the dataset

Is there a python/sklearn implementation of my idea? Right now I manually split the input depending on the value of c when training and evaluating the models.

Probably. It's the type of mistaken thing people do in data science, and Python is used a lot in data science. However, here we try to guide people into a better way of doing things.

Fit a single model on the entire dataset. Include the variable c as a fixed effect and also fit interactions with this variable and your other variables.

Improve this answer
$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.