1
$\begingroup$

I have a process with ~10 features and ~100 responses, and would like to search for models for how those features interact to create various responses. ~100 experiments were run, exploring combinations of values of the features, so it’s a limited training set. I was thinking about doing multilinear regression (possibly exploring quadratic terms) but I'm not sure what's the most elegant/simple way is, in scikit-learn, to explore all possible model parameters to find the most convincing models.

If you have experience with this sort of problem (seems like a standard subset selection problem?) in scikit-learn or statsmodels, please give me some pointers. And please let me know if my question is unclear.

Improve this question
$\endgroup$
0
1
$\begingroup$

First, I am not sure I totally understand your notation. Are you trying to say, if you divided number of data points by number of feature you get ~10, i.e., 10 times data than number of features? The big O notation without n in there is really strange..

And what do you mean by $O(10^2)$ "experiments" were run ..., How do you define "experiments"? is it number of predictions?

Without totally understand it, I will still try to answer. I would not recommend Best subset or stage wise feature selection. Try to use all features with regularization.

This is really a standard problem. Try to search ridge regression. In R glmnet can be used.

Improve this answer
$\endgroup$
2
  • $\begingroup$ Hi, thanks for chiming in. I removed the confusing notation. Yes, there are 10x more data points than features (alternatively called predictors, or variables, or simply vector X), but there are only about 3 steps along each features direction. By "responses", I meant the outcomes (also "output" or "dependent variable", or simply vector Y), not the data points. There are about ~100 output variables. I'll think about your answer and how to implement regression in this case. $\endgroup$ – DJname Oct 17 '16 at 20:45
  • $\begingroup$ Clarified the notation. There are about 10 features, 100 output variables, and 100 data points. I'll think about how to implement regularization in this problem. $\endgroup$ – DJname Oct 17 '16 at 20:51

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.