"Robust" normalization of features from multiple groups and unknown distributions prior to learning

Question

I'm working on a machine learning project involving statistical analysis (and later discriminatory classification) of different proteins (samples) drawn from multiple, potentially overlapping classes / groups, all of which are drawn from a much larger background population (all mammal proteins).

I have a list of features that I calculate for each individual protein, and then serve as the basis for classification (using machine learning) for each class / group of proteins later. (The features are continuous and numerical, but may be very different, and there's no reason to assume that the underlying distribution is normal, or related).

I want to normalize and center the "raw" calculated feature values for later training. The standard approach of normalizing by a Z-score, then centering [0,1] seems inappropiate, since there's no reason to assume the underlying distributions are normal (I have hundreds of different features - frequency counts, bigrams counts, physiochemical property values etc') .

I've heard of "robust statistical measures", and thought of first normalizing (using the medians) all the features against each other, then applying scikit's normalization+centering to the "median normalized" set of features, but I have no idea if this makes sense, or will retain the differences in the original data.

(Note - I also expect a small amount of significant outliers for different features and properties, so using the median is attractive in that regard as well) . Does this make sense? Is there a better way to normalize between all the groups (Rather than just using raw scores for the features)?

Answer 1

I agree with gaborous in that you maybe seeking standardization, and a quick search for a visualization reveals lectures on why standardization is useful.

TLDW version is that this is all to avoid squashed (elliptical or canoe) shaped input space, because such shapes will cause linear regression to wander.

---

Credit to Blitz Kim

---

Now from what little I've grokked of the fields concerned it seems as though one can get away with ranges that do not produce a nice dartboard like shapes, eg -1 to 1 and 0 to 3, however, ranges greater than 1 or less than -1 may cause high variability in outputs, or lead to the exploding or vanishing gradient problem.

Supposedly there are activation functions and other methods that can be used that are more resilient or less likely to be susceptible to gradients misbehaving, but when a little data massaging can mitigate having a later fight with debugging a model, it seems like a far better choice to pre-clean the input space.

---

As to what makes sense or not... personally when reading (and re-reading) the question my instinct would be to suggest adapting something like word2vec for the data to input encoding method and feeding that to a GCN for classification and perhaps generation.

Because supposedly word to vector methods better preserve relationships, like similar meanings of words, or perhaps in this case similar functions between proteins that may seem unrelated at first glance in structure. And Graph Convolution Networks, because supposedly they do really well at point cloud inputs, which is kinda what word2vec methods output

But this is just based off speculation on what it is that you're attempting to do, and what I've learned on these subjects. Essentially I'd treat proteins as paragraphs because they express that level of complex meaning/intent.