0
$\begingroup$

I have a dataset as follows:

mydata:
    f1,f2,f3, ..., fn, target
 s1 34,56,32,...., 43,   0
 s2 37,60,33,...., 54,   1
      ....  
 sm 89,86,56,...., 90,   0

I did some feature engineering processes and created new feature for each attribute as follows:

 my_newdata:
    f1,f1_new, f2, f2_new, f3, f3_new, ...,  fn, fn_new, target
 s1 34,  3,    56,   5   , 32,   6   , ..., 43,   3    , 0
 s2 37,  5,    60,   12  , 33,   8   , ..., 54,   1    , 1
                ....  
 sm 89,  6,    86,  12   , 56,   2   , ..., 90,   4    , 0

Basically, for any feature, I created and extracted a new feature. My problem is a classification task so I trained the model (using the new data-set) with a classifier and obtained 73% accuracy value. Now, I want to measure the impact/influence of the new features to the model performance. My question is, Is there any statistical test or way to show how valuable this new features are for the prediction performance. I read about feature interaction to the model performance etc., enter link description here

but I am not sure feature interaction is the right experiment to show the impact of new feature to the performance of the model. Any idea to show this impact?

Improve this question
$\endgroup$
3
  • 2
    $\begingroup$ Train one model on the original features, train another model on the engineered features, compare the results? Is that what you want? Or to compare the original features model with the original+engineered features model? $\endgroup$
    – user2974951
    Commented Mar 5, 2021 at 13:01
  • $\begingroup$ yes,this could be a solution, but I was wondering if there is a statistical way to compare and measure the impact? $\endgroup$
    – Spedo
    Commented Mar 6, 2021 at 8:23
  • $\begingroup$ This really depends on the model. If the model is a regression model then you might be able to use an ANOVA to compare two or more nested models. But I wouldn't bring statistical testing in this procedure. $\endgroup$
    – user2974951
    Commented Mar 8, 2021 at 21:43

0

Reset to default

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.