Featurization before or after dataset splitting I was reading this post about a reproducibility tool when I realized that the author decided to split the train/test datasets before featurization.
It didn't make sense to me, because why would you not be able to commute those operations? But just a few days later I found this comment in a deepchem repo issue:

In cases of data augmentation, there are use-cases when it might make sense to featurize data after it has been split into train/test/validation.

And I have no idea how that would be a thing, but apparently it is. What kind of data augmentation - or any other data transformation for that matter - could justify postponing the dataset splitting?
 A: That comment is correct: we have to do "feature extraction" from our training data only. 
Let's consider one of the most common data-transformation procedures, centring. We get an "expected value" $\hat{\mu}_{x_j}$ for our feature $x_j$ and then we subtract that from the values of $x_j$, nothing magical. A central question is: what this "expected value" reflects; does it reflect our understanding of $x_j$ using the whole sample or just the training sample? If we use the whole sample we have what is called data-leakage, "we cheat" using information that should be available during prediction. To give an NLP example, if some very unusual word or n-gram is present in our corpus and all instances happen to land in the test set, it is obviously wrong to inform the process of convert our collection of text documents to a matrix of token counts with that unusual n-gram. It will give us a false scene of security about the generalised performance of our procedure. Therefore, when doing serious feature extraction/engineering we need to use only the training data and not the whole dataset. 
