# Feature varying in orders of magnitude

I have learned that you are supposed to scale features as a preprocessing step for most ML algorithms (so that all features are of the same magnitude and no direction is preferred, for example in Gradient Descent).

But what is the best approach if one feature varies between orders of magnitude, i.e. meaningful values for some feature $f$ are $f\in\{0, 1, 2, 3, 4, 100, 101, 102, 100000, 100001, 100002\}.$

If I scale, I lose all meaning of difference between for example 0,1,2,3 and 4. If I take the logarithm (as to alleviate the differenc between 100000 and the rest), I have the same problem.

• Do I derive correctly that this feature is thought to be continuous? (If not, you could probably just use it as categorial feature instead.) If yes: you might still be able to derive some further information from it, e.g. if you have feature values only from certain ranges (0-4, 100-200, 1000000-200000, etc). – geekoverdose Jun 30 '16 at 14:19
• In my case, it just happens to have values like above (think of the length of some metal component for machines, some are really tiny, some are extraordinary big). But their "impact" is not directly linearly related to their size: A tiny intricate metal part might be pretty expensive in comparison to something big, but in the same order of magnitude the effect would be linear. For example: Tiny machine part = 10\$. Slightly bigger machine part (still tiny) = 15 \$. Large machine part = 10\$, very large machine part = 15 \$. – Mercury Bench Jun 30 '16 at 14:24
• So it is linearly proportional in some lengthscale and then "resets" up until the next order of magnitude. – Mercury Bench Jun 30 '16 at 14:27
• OK, this sounds like there is additional information comprised in that variable. If e.g. splitting this variable into multiple variables (which you could scale properly) would be an option for you I'll try to formulate a short answer for this. – geekoverdose Jun 30 '16 at 14:33