I'm unsure about which is the best approach for feature selection for Isolation Forest. I am using a dataset which initially has 5 numerical columns and 50 categorical. After pre-processing and feature engineering I end up with 9 numerical columns and by using one-hot-encoding I end up with 70 binary columns. Based on a paper, including categorical attributes is not a good idea for IForest - but that is mostly dataset-specific. I'm reading the 2012 Isolation Forest paper (https://cs.nju.edu.cn/zhouzh/zhouzh.files/publication/tkdd11.pdf) and at some point the author says:
In this experiment, we study a special case of high dimensional data in which data sets have a large number of irrelevant attributes. We show that iForest has a significant advantage in processing time. We simulate these high dimensional data sets using the first thirteen data sets introduced in Table III. For each data set, uniformly distributed random attributes, valued between 0 and 1 are added. Such that, there is a total of 512 attributes in each data set. We use a simple statistical test, Kurtosis [Joanes and Gill 1998], to select an attribute subspace from the sub-sample before constructing each iTree. Kurtosis measures the ‘peakness’ of a univariate distribution. Kurtosis is sensitive to the presence of anomalies and hence it is a good attribute selector for anomaly detection. After Kurtosis has provided a ranking for each attribute, a subspace of attributes is selected according to this ranking to construct each tree.
But it is not clear to me how they used Kurtosis for feature selection. Although I understand that they state they use it on the sub-samples, I tried it on my dataset as a whole. Hence, I used scipy's kurtosis for each column and then ranked their values. It seems that the quoted text suggests that they picked the highest valued kurtosis attributes. But I'm not sure about it. In my test, if I pick an arbitrary number of attributes in descending order then I simply get awful results. Whereas when I pick in ascending order I get quite good results. But this seems to contradict the author above. Furthermore, by using domain knowledge or just common sense the ones that have higher kurtosis seems completely useless.
Any clarifications are appreciated. Also, any other suggestion for feature selection.