According to Wikipedia, the only correct interpretation of kurtosis is "tail extremity," the logic being that datapoints within one standard deviation of the mean are raised to the fourth power and don't contribute to the expected value $\mathbb{E}(\frac{X-\mu}{\sigma})^4$ since they are much less than 1.
As Westfall (2014) notes, "...its only unambiguous interpretation is in terms of tail extremity; i.e., either existing outliers (for the sample kurtosis) or propensity to produce outliers (for the kurtosis of a probability distribution)." The logic is simple: Kurtosis is the average (or expected value) of the standardized data raised to the fourth power. Any standardized values that are less than 1 (i.e., data within one standard deviation of the mean, where the "peak" would be), contribute virtually nothing to kurtosis, since raising a number that is less than 1 to the fourth power makes it closer to zero. The only data values (observed or observable) that contribute to kurtosis in any meaningful way are those outside the region of the peak; i.e., the outliers. Therefore, kurtosis measures outliers only; it measures nothing about the "peak".
If that's the reasoning, wouldn't higher moments be a better measure of tailed-ness? E.g. if we take $\mathbb{E}(\frac{X-\mu}{\sigma})^6$, the points within one standard deviation of the mean would contribute even less and points outside would contribute even more. Is there any formal reasoning for why the 4th power is the "correct" one, and not the 6th, 10th, 1000th?