I have often heard the argument that in higher dimensions: the "mode" (most common value) of a probability distribution function does not correspond to the "expectation" (mean) of the probability distribution function. The implication of this argument is that it becomes less meaningful to calculate the "mode" of a probability distribution function (i.e. the Bayesian MAP estimate), but rather it now becomes more advantageous to calculate the "expected value" of the probability distribution function (e.g. via MCMC).
I am trying to better understand this logic:
I have heard that in high dimensional space, (it can be mathematically proven that) the majority of the volume of a hypersphere becomes concentrated around a thin outer ring of the hypersphere. If you apply this metaphor to high dimensional data, this means that the majority of the data is located around the extremities of the space, whereas most of the space itself is likely to be "sparsely populated". I can understand this argument. (e.g. Imagine an onion ring (thin torus) - the data would be the "onion crust" and all the volume in the middle is empty)
What I don't understand is the following: In high dimensional space, why should the difference between the "mode" and the "expectation" of the probability distribution function be any more different than in low dimensional space? In high dimensional space, why is the "mode" not as useful (in terms of information, in terms of describing the probability distribution) as the "expectation"? In high dimensional space, logic tells us that most of the space is empty and the data is concentrated around the exterior. One could make the argument that "most of the data is located around the exterior", therefore the "mode" and "expectation" should be both as informative?
I have attached this picture as reference: https://i.stack.imgur.com/fOGE6.jpg
Can someone please help me better understand this?