Combining features to improve F1 score in random forest

Question

In a project we have to use random forests to classify data into 5 sets. The data set is large (300k data elements and 9 features) and the result is based of F1 score rather than mean square error. A suggested method is to use new features apart from the given 9 features. Can some one please suggest why would taking combinations of features like X1+X2*X3^2 where Xi is a feature improve the accuracy? I am skeptical since all the trees in random forest are already grown to full depth.

Answer 1

Creating new (composite) features as functions of existing terms can capture interactions between the variables not readily apparent which could otherwise be missed growing the trees. Also, since each tree in a random forest is built from a different sampled subset of the available variables, adding composite variables to your dataset effectively increases the probability that one of the variables used to build the composite variable will have an opportunity to contribute information to a tree in the forest.

For example, if you have a dataset comprised of the variables W, X, Y, Z and you build a random forest classifying Z ~ W + X + Y, the random forest algorithm will build a few trees that leave out the Y variable, such as Z ~ W + X. If you built a composite variable P=X/Y and built your forest classifying Z ~ W + X + Y + P, then some trees that leave out Y will still include P, so Y's information still contributes to the model. This effectively adds some additional weight to the variables used to build the composite variables, since they are more likely to contribute to models.

Answer 2

Well assume the true class is sign (X1+X2*X3^2) (2 class problem)? How would you create that with a [deterministic] tree? basically with each classifier think about what sort of decision functions it can naturally make and add any relevant features that it can't.