When doing predictions with Random Forests, we very often (or always) need to perform some pre-processing. Since I have a background of Computing and pretty much all I know from statistics comes from self-learning, this process becomes more intuition and less theory.
For instance, some of the things I get stuck with is dealing with
- Outliers. Should we remove them all? If so, we consider an outlier based on the 3/2 rule? Should we keep them? Why?
- When dealing with deltas of observations (as an example, suppose I'm subtracting a student grade from another), should I normalize the delta of all students or just stick to the absolute delta?
- Sticking to the same student case, If I have cumulative data (suppose for every test I sum their last grades). Should the process be the same?
- Do we need to apply any data transformation like
logor any other? If so, when should it be done? When data range is large? What's the point of changing the domain of the data here?
- If I have a Categorical target, can I apply regression instead of classification so the output would be (suppose the classes are 0, 1, 2) 0.132, 0.431. Would it be more accurate?
- In what kind of problems is Random Forest more indicated? Large datasets?
- Should I discard the less important variables? Maybe it just creates noise?
I know the pre-processing depends on the problem, data, etc. and I know there are a lot more things to look for when pre-processing. Here I'm more trying to understand the concepts behind pre-processing the data and the key points to look for when doing so. So with that in mind, what would be the key points to look for when pre-processing data? (If I didn't mention any other important point, and I'm sure a lot is missing, please consider that too). Imagine you're teaching that to your grandpa :)