In feature engineering, should I drop all features that can be calculated using other features?
For example, let us say that we have this dataset:
f1 f2 f3 f4 target
1 4 0.25 5 9
3 6 0.5 9 8
5 8 0.625 13 12
2 8 0.25 10 15
In this dataset,
f3 = f1 / f2
f4 = f1 + f2
Should we drop f3 and f4?
Does the decision depend on the size of the dataset?
Does the decision depend on the algorithm used (Random Forest, Neural Network, etc)?
Does the decision depend on the machine-learning type: regression, classification, etc?
Does the decision depend on the formula that links the features? For example, the formula that links
f3
tof1
andf2
is simple division.
Can you please also recommend resources to learn more about this topic?