Why is the Manhattan distance (or block distance) appropriate when I have a discrete data set?

Why is the Manhattan distance (or block distance) appropriate when I have a discrete data set and the Euclidean distance is appropriate when I have continuous numerical variables?

• What do you call discrete data for you? Are that data values scale (with rough, discreet gauge though, the bins) or truly categorical (nominal or ordinal)? Feb 6 '17 at 7:24

I wouldn't always agree with that statement. but here is an intuition why probably the default should be this way.

• Euclidean distance is 'as the crow flies'. It is the shortest path if you continuously linear interpolate between these points. The distance from 0 to (1,1) passes through (0.5,0.5) and the distance is $\sqrt{2}$.
• Manhattan distance assumes independent attributes. In particular, no continuous linear independence. If you have a discrete variable, Manhattan is the number of steps you have to do there.

But that is just an intuition, a "rule of thumb"

For example a discrete histogram, I'd rather use a divergence distance.

Or if you consider "money" to be discrete (I would not suggest to do this here), and the number of products bought. There will usually be some linear dependence between these attributes, then Euclidean is likely more appropriate (not that it would make any sense to apply a distance function on quantity,cost without careful feature engineering.)

• independent attributes in what sense? Feb 6 '17 at 7:31
• In the 'intuition' sense. But e.g. linear dependence of attributes is a reason to not use Manhattan, even on discrete data! Feb 6 '17 at 7:34
• I doubt (or have difficulty to understand/appreciate) for that intuitive account of "independence". (To me, whether euclidean or manhattan distances are usable or valid depends, first of all, on the nature of the data scale. It is discrete scale? or categorical? or counts? These are different cases. But the OP has not clarified it in the question) Feb 6 '17 at 8:12
• I couldn't find the source for that. Could have been a source related to Gowers. But maybe it convinces you that Euclidean is rotation invariant? So it "does not matter" if you rotate in attributes, so they are not independent. Feb 6 '17 at 23:24
• The notion of rotation is correct. I would rather call that something like "Euclidean distance exploits the isotropic, smooth side of the space" in place of "dependency". With categorical, e.g. binary attributes, using euclidean distance may be still valid, perhaps, if we don't insist that the distance be isotropic, that it should remain meaningful when we rotate the data cloud. That is, $\sqrt{2}$ is not unsenseless with binary attributes - if not require it to survive the rotation. Feb 7 '17 at 3:37