The kurtosis is to measure the peakedness and flatness of a distribution. The density function of the distribution, if it exists, can be viewed as a curve, and has geometric features (such as curvature, convexity, ...) related to its shape.
So I wonder whether the kurtosis of a distribution is related to some geometric features of the density function, which can explain the geometric meaning of kurtosis?
The moments of a continuous distribution, and functions of them like the kurtosis, tell you extremely little about the graph of its density function.
Consider, for instance, the following graphs.
Each of these is the graph of a non-negative function integrating to $1$: they are all PDFs. Moreover, they all have exactly the same moments--every last infinite number of them. Thus they share a common kurtosis (which happens to equal $-3+3 e^2+2 e^3+e^4$.)
The formulas for these functions are
$$f_{k,s}(x) = \frac{1}{\sqrt{2\pi}x} \exp\left(-\frac{1}{2}(\log(x))^2\right)\left(1 + s\sin(2 k \pi \log(x))\right)$$
for $x \gt 0,$ $-1\le s\le 1,$ and $k\in\mathbb{Z}.$
The figure displays values of $s$ at the left and values of $k$ across the top. The left-hand column shows the PDF for the standard lognormal distribution.
Exercise 6.21 in Kendall's Advanced Theory of Statistics (Stuart & Ord, 5th edition) asks the reader to show that these all have the same moments.
One can similarly modify any pdf to create another pdf of radically different shape but with the same second and fourth central moments (say), which therefore would have the same kurtosis. From this example alone it should be abundantly clear that kurtosis is not an easily interpretable or intuitive measure of symmetry, unimodality, bimodality, convexity, or any other familiar geometric characterization of a curve.
Functions of moments, therefore (and kurtosis as a special case) do not describe geometric properties of the graph of the pdf. This intuitively makes sense: because a pdf represents probability by means of area, we can almost freely shift probability density around from one location to another, radically changing the appearance of the pdf, while fixing any finite number of pre-specified moments.
[NB this was written in response to another question on site; the answers were merged to the present question. This is why this answer seems to respond to a differently worded question. However much of the post should be relevant here.]
Kurtosis doesn't really measure the shape of distributions. Within some distribution families perhaps, you can say it describes the shape, but more generally kurtosis doesn't tell you terribly much about the actual shape. Shape is impacted by many things, including things unrelated to kurtosis.
If one does image searches for kurtosis, quite a few images like this one show up:
which instead seem to be showing changing variance, rather than increasing kurtosis. For comparison, here's three normal densities I just drew (using R) with different standard deviations:
As you can see, it looks almost identical to the previous picture. These all have exactly the same kurtosis. By contrast, here's an example that is probably nearer to what the diagram was aiming for
The green curve is both more peaked and heavier tailed (though this display isn't well suited to seeing how much heavier the tail actually is). The blue curve is less peaked and has very light tails (indeed it has no tails at all beyond $\sqrt{6}$ standard deviations from the mean).
This is usually what people mean when they talk about kurtosis indicating the shape of the density. However, kurtosis can be subtle -- it doesn't have to work like that.
For example, at a given variance higher kurtosis can actually occur with a lower peak.
One must also beware the temptation (and in quite a few books it's openly stated) that zero excess kurtosis implies normality. There are distributions with excess kurtosis 0 that are nothing like normal. Here's an example:
Indeed, that also illustrates the previous point. I could readily construct a similar-looking distribution with higher kurtosis than the normal but which is still zero at the center - a complete absence of peak.
There are a number of posts on site that describe kurtosis further. One example is here.
For symmetric distributions (that is those for which the even centred moments are meaningful) kurtosis measures a geometric feature of the underlying pdf. It is not true that kurtosis measures (or is in general related) to the peakedness of a distribution. Rather, kurtosis measure how far the underlying distribution is from being symmetric and bimodal (algebraically, a perfectly symmetric and bimodal distribution will have a kurtosis of 1, which is the smallest possible value the kurtosis can have)[0].
In a nutshell[1], if you define:
$$k=E(x-\mu)^4/\sigma^4$$
with $E(X)=\mu,V(X)=\sigma^2$, then
$$k=V(Z^2)+1\ge1$$
for $Z=(X-\mu)/\sigma$.
This implies that $k$ can be seen as a measure of dispersion of $Z^2$ around its expectation 1. In other words, if you have a geometrical interpretation of the variance and the expectation, than that of the kurtosis follows.
[0] R. B. Darlington (1970). Is Kurtosis Really "Peakedness?". The American Statistician , Vol. 24, No. 2.
[1] J. J. A. Moors (1986).The Meaning of Kurtosis: Darlington Reexamined. The American Statistician, Volume 40, Issue 4.
A different kind of answer: We can illustrate kurtosis geometrically, using ideas from http://www.quantdec.com/envstats/notes/class_06/properties.htm: graphical moments.
Start with the definition of kurtosis: $$ \DeclareMathOperator{\E}{\mathbb{E}} k = \E \left( \frac{X-\mu}{\sigma} \right)^4 =\int \left(\frac{x-\mu}{\sigma}\right)^4 f(x) \; dx $$ where $f$ is the density of $X$, $\mu, \sigma^2$ respectively expectation and variance. The nonnegative function under the integral sign integrates to the kurtosis, and gives contribution to kurtosis from around $x$. We can call it the kurtosis density, and plotting it shows the kurtosis graphically. (Note that in this post we are not using the excess kurtosis $k_e=k-3$ at all).
In the following I will show a plot of graphical kurtosis for some symmetric distributions, all centered at zero and scaled to have variance 1.
Note the virtual absence of contribution to the kurtosis from the center, showing that kurtosis does not have much to do with "peakedness".
Kurtosis is not related to the geometry of the distribution at all, at least not in the central portion of the distribution. In the central portion of the distribution (within the $\mu \pm \sigma$ range) the geometry can show an infinite peak, a flat peak, or bimodal peaks, both in cases where the kurtosis is infinite, and in cases where the kurtosis is less than that of the normal distribution. Kurtosis measures tail behavior (outliers) only. See https://www.ncbi.nlm.nih.gov/pmc/articles/PMC4321753/
Edit 11/23/2018: Since writing this post, I have developed some geometrical perspectives on kurtosis. One is that excess kurtosis can indeed be visualized geometrically in terms of deviations from the expected 45 degree line in the tails of the normal quantile-quantile plot; see Does this Q-Q plot indicates leptokurtic or platykurtic distribution?
Another (perhaps more physical than geometrical) interpretation of kurtosis is that kurtosis can be visualized as the point of balance of the distribution $p_V(v)$, where $V = \{(X - \mu)/\sigma \}^4$. Note that (non-excess) kurtosis of $X$ is equal to $E(V)$. Thus, the distribution of $V$ balances at the kurtosis of $X$.
Another result that shows that geometry in the $\mu \pm \sigma$ range is nearly irrelevant to kurtosis is given as follows. Consider the pdf of any RV $X$ having finite fourth moment. (Thus the result applies to all empirical distributions.) Replace the mass (or geometry) within the $\mu \pm \sigma$ range arbitrarily to get a new distribution, but keep the mean and standard deviation of the resulting distribution equal to $\mu$ and $\sigma$ of the original $X$. Then the maximum difference in kurtosis for all such replacements is $\le 0.25$. On the other hand, if you replace the mass outside the $\mu \pm \sigma$ range, keeping the center mass as well as $\mu$, $\sigma$ fixed, the difference in kurtosis is unbounded for all such replacements.
