I'm normalizing multivariate numeric data that has both negative and positive values. For the sake of the question let's assume a range of e.g. $[-10000,10000]$ with a lot of values in $[-1,1]$. I've come up with a normalization that seems to work for me (details below), and now I wonder:
- if this normalization actually has a well known name that I could use to explain it, and
- if any other normalization approaches come to mind that seem suitable for my situation, then I'd also be happy to hear about them.
My data is mostly centered around the origin (looks a bit like Gaussian with $\mu=0$, but some variables are definitely skewed or else). For this question I now consider only one variable $x$ of those variables, as the problem seems to apply all variables alike. When I add class information to my observation - which is the subsequent prediction target, hence I cannot use in during prediction - then I see that a small subset of classes cause huge outliers in $x$. The spread of their underlying distribution seems to be way larger than the spread of the distribution of other classes. Standard normalization, especially the scaling part, like $mean=0, std=1$, or $median=0, mad=1$, leaves the majority of samples of all those non-outlier classes in a tiny spot in the origin.
Examples: this is an excerpt of a larger scatterplot matrix (subset of samples only) containing outliers of one of those classes. The values could all be $<-1$, in $[-1,1]$, or $>1$, or a mix of all of those. The origin is most likely to be around the center of the samples of the other classes.
My thoughts so far:
- I would not want to use some thresholding or similar to get rid of those values, I would be rather interested in using a clever normalization for this case instead.
- My first intuition was that I could solve a problem like this using a logarithm. However, as I have lots of values in $[-1, 1]$ I would get rather small log transformations of those. Further, mapping negative values is problematic (I already need the sign for values $<0$). If I would add $max(x)$ to $x$ before that operation, then in case of the outlier values being originally negative, I would still have the majority of samples in a tiny spot towards the upper boundary of the transformed $x$. In that case the $log$ actually makes things worse.
After a bit of thinking and basic math my first approach for this normalization is now:
$y = sign(x) \cdot \sqrt[k]{|x|}$, with $k > 1$
Effect for $k=2$:
Strictly monotonic (which I guess I really need for my data), non-linear in the complete possible range of values $[-\inf,\inf$], nicely normalizes outliers in a non-linear way towards the origin, and behaves more linearly closer to the origin.
Questions:
- This seems so obvious to me that I guess there is a well known name for this normalization, or that I overlook that I can transform that with basic math into something well known (Box-Cox or alike), or that I overlook problems with what I do here (e.g. de-skewing-effect not strong enough, compared to other normalization approaches). Can anyone shed some light on this?
- Additional to 1., I would also be interested in other suggestions for normalization approaches to use in this situation, should any come to mind.
