Feature selection vs Feature subset selection I am trying to verify if these two terms "Feature selection" and "Feature subset selection" relate to the same term. After viewing a number of articles it seems that both are used interchangeably, while I was initially assuming that the latter relates to ensemble feature selection where not a single feature subset but a number of feature subsets is calculated.
Can you confirm that these two are just different names for the same task or is there some significant difference similar to my initial assumptions?
 A: I like "An Introduction to Statistical Learning with Applications in R" by G.James et al., so I took a look at this book.
According to the book, it seems like feature selection is more wide term that includes methods of subset selection. P. 218:
"In this chapter, we see some approaches for automatically performing feature selection or variable selection — that is, for excluding irrelevant variables from a multiple regression model.
<...>
• Subset Selection. This approach involves identifying a subset of the p
predictors that we believe to be related to the response. We then fit
a model using least squares on the reduced set of variables.
• Shrinkage. This approach involves fitting a model involving all p predictors.However, the estimated coefficients are shrunken towards zero
relative to the least squares estimates. This shrinkage (also known as
regularization) has the effect of reducing variance. Depending on what
type of shrinkage is performed, some of the coefficients may be estimated
to be exactly zero. Hence, shrinkage methods can also perform
variable selection".
So it seems that for feature selection we may use subset selection (i.e. best subset selection, backward or forward selection) or shrinkage methods such as lasso regression.
Hope this helps.
