Return to Question

Assume that we have 30 features each of which can potentially influence the result. I try to develop a model that uses some of the features. Each feature can be either considered or not. It means that in total we have 2^30 combinations of what arguments to use. I would like to notice that model is fixed, I do not change the model (in a broad sense) I just decide what variables to use.

Now we have considered several thousand combinations and for each of the combinations we have a measure of how good it is (for that, of course, we use the testing set). For example, we know that if we take variable 1, 7, 13, and 27 we get an error (accuracy of the model) that is equal to 123.456. If we use variable 3, 4, 10, 11, 13, 27 and 29. We get an error equal to 23.456.

My question is: How to summarize the knowledge about performance of the model as a function of the used arguments?

In other words I need a "meta-model" that describes performance of my model. I need this because I want to use the obtained knowledge to search more intelligently over the huge space of potential models.

More specifically, I have a mapping from N binary vectors. Each binary vector indicates if a corresponding variable is used or not. We have a mapping from binary vectors to real number describing performance of correspondent model. Now I want to use this meta-date to train a meta-model. But before I start a meta-training I would like to decide what model is a good one to train. This model should capture such concepts as "information stored in variables". For example I can imagine that variable 7 has no additional information to the combination of variables 3 and 5.

Assume that we have 30 features (inputs) each of which can potentially influence the result (output). We try to use the available ("observed") mapping from features to targets to develop a predictive model that uses only some of the features. Each feature can be either included into the input of the model or not. It means that in total we have 2^30 combinations of what arguments to use. I would like to notice that model is fixed, I do not change the model (in a broad sense), I just decide what variables to use.

Now we have considered several thousand combinations and for each of the combinations we have a measure of how good it is (for that, of course, we use the testing set). For example, we know that if we take variable 1, 7, 13, and 27 we get an error (accuracy of the model) that is equal to 123.456. If we use variable 3, 4, 10, 11, 13, 27 and 29. We get an error equal to 23.456.

My question is: How to summarize the knowledge about performance of the model as a function of the used arguments?

In other words I need a "meta-model" that describes performance of my models. I need this because I want to use the obtained knowledge to search over the huge space of potential models more intelligently.

More specifically, I have a mapping from N binary vectors to a real (float) value. Each binary vector indicates if a corresponding variable is used or not and the real output describes the performance of the corresponding model. Now I want to use this "meta-data" to train a "meta-model". But before I start a "meta-training" I would like to decide what model is a good one to train. This model should capture such concepts as "information stored in variables". For example I can imagine that variable 7 has no additional information to the combination of variables 3 and 5. Or, we can have "orthogonal" variables that cover the target space "independently".

Assume that we have 30 features each of which can potentially influence the result. I try to develop a model that uses some of the features. Each feature can be either considered or not. It means that in total we have 2^30 combinations.

Now we have considered several thousand combinations and for each of the combinations we have a measure of how it good. For example, know that if we take variable 1, 7, 13, and 27 we get an error (accuracy of the model) that is equal to 123.456. If we use variable 3, 4, 10, 11, 13, 27 and 29. We get an error equal to 23.456.

Now I would like to summarize this knowledge to be able to predict how use of arguments changes results. How can it be done?

Assume that we have 30 features each of which can potentially influence the result. I try to develop a model that uses some of the features. Each feature can be either considered or not. It means that in total we have 2^30 combinations of what arguments to use. I would like to notice that model is fixed, I do not change the model (in a broad sense) I just decide what variables to use.

Now we have considered several thousand combinations and for each of the combinations we have a measure of how good it is (for that, of course, we use the testing set). For example, we know that if we take variable 1, 7, 13, and 27 we get an error (accuracy of the model) that is equal to 123.456. If we use variable 3, 4, 10, 11, 13, 27 and 29. We get an error equal to 23.456.

My question is: How to summarize the knowledge about performance of the model as a function of the used arguments?

In other words I need a "meta-model" that describes performance of my model. I need this because I want to use the obtained knowledge to search more intelligently over the huge space of potential models.

More specifically, I have a mapping from N binary vectors. Each binary vector indicates if a corresponding variable is used or not. We have a mapping from binary vectors to real number describing performance of correspondent model. Now I want to use this meta-date to train a meta-model. But before I start a meta-training I would like to decide what model is a good one to train. This model should capture such concepts as "information stored in variables". For example I can imagine that variable 7 has no additional information to the combination of variables 3 and 5.

Assume that we have 30 features each of which can potentially influence the result. I try to develop a model that uses some of the features. Each feature can be either considered or not. It means that in total we have 30^2 combinations.

Now we have considered several thousand combinations and for each of the combinations we have a measure of how it good. For example, know that if we take variable 1, 7, 13, and 27 we get an error (accuracy of the model) that is equal to 123.456. If we use variable 3, 4, 10, 11, 13, 27 and 29. We get an error equal to 23.456.

Now I would like to summarize this knowledge to be able to predict how use of arguments changes results. How can it be done?

Assume that we have 30 features each of which can potentially influence the result. I try to develop a model that uses some of the features. Each feature can be either considered or not. It means that in total we have 2^30 combinations.

Now we have considered several thousand combinations and for each of the combinations we have a measure of how it good. For example, know that if we take variable 1, 7, 13, and 27 we get an error (accuracy of the model) that is equal to 123.456. If we use variable 3, 4, 10, 11, 13, 27 and 29. We get an error equal to 23.456.

Now I would like to summarize this knowledge to be able to predict how use of arguments changes results. How can it be done?