I was recently doing some reading on feature selection and had a little doubt on this topic . The article on which my question is based on reading this article : http://machinelearningmastery.com/an-introduction-to-feature-selection/ .
Say I have my dataset splitted into 3 parts namely : Training , Validation and Testing . I want to apply the univariate feature selection method namely , the Chi Square Test and Anova Test . Now , the article states that feature selection methods should be applied on the Validation Set rather than the Training Set . But shouldn't this be the case when dealing with Wrapper Methods ( such as Recursive Feature Elimination ) and Embedded Methods ( Lasso , Ridge Regression ) ? For the univariate tests , shouldn't the pipeline be something like this :
Training Set with all features --> Apply Statistical Test --> Remove Insignificant Variables --> Train Model with Selected Features .