8
$\begingroup$

Take a look at the image below. Blue line indicates standard normal pdf. The red zone is supposed to be equal to the sum of grey areas (sorry for awful drawing).

I wonder can we create a new distribution with higher peak by shifting grey zones to the top (red zone) of the normal pdf?

new distribution with higher peak

If such transformation can be made, than what do you think about the kurtosis of this new distribution? Leptokurtic? But it has the same tails as the normal distribution does! Undefined?

$\endgroup$
  • 1
    $\begingroup$ The question is handsome but the drawing is indeed awful. The more sharp-kurtic-than-normal distribution is supposed to be heavier-tailed. But you didn't draw these tail regions (which also should be coloured red). What their areas you suppose to add up to? $\endgroup$ – ttnphns Dec 27 '13 at 12:17
  • 1
    $\begingroup$ Why not try it? Simulate (say) 10,000 from a standard normal, then move some numbers to make the distribution that you want. Then you could draw the line with a program and compute the kurtosis as well. $\endgroup$ – Peter Flom - Reinstate Monica Dec 27 '13 at 12:32
  • $\begingroup$ If you are prepared to sacrifice differentiability of the density, then you could construct such a distribution (which would have a piece-wise density). $\endgroup$ – Alecos Papadopoulos Dec 27 '13 at 13:05
  • 2
    $\begingroup$ @ttnphns, sorry if the tag misled you. I hoped that picture would make it clear that I don't want any changes in the tails. Usually, textbooks discuss kurtosis comparing the simultaneous change in the peak and the tails. I want to understand what can be said about kurtosis when only peak becomes higher. $\endgroup$ – Yal dc Dec 27 '13 at 15:33
  • 1
    $\begingroup$ Yal dc - you should note that your standard deviation has changed, so the 'tails' aren't the same unless you use some particular definitions of tail $\endgroup$ – Glen_b -Reinstate Monica Dec 27 '13 at 20:48
12
$\begingroup$

There will be an infinite number of distributions that look very similar to your drawing, with a variety of different values for kurtosis.

With the particular conditions in your question and given we hold the crossover point to be inside, or at least not too far outside $\pm 1$, it should be the case that you get a slightly larger kurtosis than for the normal. I will show three cases where that happens, and then I'll show one where it is smaller -- and explain what causes it to happen.

Given that $\phi(x)$ and $\Phi(x)$ are the standard normal pdf and cdf respectively, let's write ourselves a little function

$$f(x) = \begin{cases} \phi(x) &\mbox{;}\quad |x| > t \\ a+b.g(x) & \mbox{;}\quad |x| ≤ t \end{cases} \ $$

for some continuous, symmetric density $g$ (with corresponding cdf $G$), with mean $0$, such that $b = \frac{\Phi(t)\, –\, ½\, –\, t.\phi(t)}{G(t)\, –\, ½\, –\, t.g(t)}$ and $a = \phi(t)-b.g(t)$.

That is, $a$ and $b$ are chosen to make the density continuous and integrate to $1$.

Example 1 Consider $g(x) = 3\, \phi(3x)$ and $t=1$,

enter image description here

which looks something like your drawing, here generated by the following R code:

f <- function(x, t=1,
              dg=function(x) 2*dnorm(2*x),
              pg=function(x) pnorm(2*x),
              b=(pnorm(t) - 0.5 - t*dnorm(t))/ (pg(t) - 0.5 - t*dg(t)),
              a=dnorm(t)-b*dg(t) ) {
       ifelse(abs(x)>t,dnorm(x),a+b*dg(x))
     }

f1 <- function(x) f(x,t=1,dg=function(x) 3*dnorm(3*x),pg=function(x) pnorm(3*x))
curve(f1,-4,4,col=2)
lines(x,dnorm(x),col=3)

Now the calculations. Let's make a function to evaluate $x^pf_1(x)$:

fp <- function(x,p=2) x^p*f1(x)

so we can evaluate the moments. First the variance:

 integrate(fp,-Inf,Inf)  # should be just smaller than 1
0.9828341 with absolute error < 1.4e-07

Next the fourth central moment:

 integrate(fp,-Inf,Inf,p=4) # should be just smaller than 3
2.990153 with absolute error < 8.3e-06

We need the ratio of those numbers, which should have about 5 figure accuracy

 integrate(fp,-Inf,Inf,p=4)$value/(integrate(fp,-Inf,Inf)$value^2)
[1] 3.095515

So the kurtosis is about 3.0955, slightly larger than for the normal case.

Of course we could compute it algebraically and get an exact answer, but there's no need, this tells us what we want to know.


Example 2 With the function $f$ defined above we can try it for all manner of $g$'s.

Here's the Laplace:

library(distr)
D <- DExp(rate = 1) 
f2 <- function(x) f(x,t=1,dg=d(D),pg=p(D))
curve(f2,-4,4,col=2)
lines(x,dnorm(x),col=3)

enter image description here

fp2 <- function(x,p=2) x^p*f2(x)


 integrate(fp2,-Inf,Inf)  # should be just smaller than 1
0.9911295 with absolute error < 1.1e-07
 integrate(fp2,-Inf,Inf,p=4) # should be just smaller than 3
2.995212 with absolute error < 5.9e-06
 integrate(fp2,-Inf,Inf,p=4)$value/(integrate(fp2,-Inf,Inf)$value^2)
[1] 3.049065

Unsurprisingly, a similar result.


Example 3: Let's take $g$ to be a Cauchy distribution (a Student-t distribution with 1 d.f.), but with scale 2/3 (that is, if $h(x)$ is a standard Cauchy, $g(x) = 1.5 h(1.5 x)$, and again set the threshold, t (giving the points, $\pm t$, outside which we 'switch' to the normal), to be 1.

dg <- function(x) 1.5*dt(1.5*x,df=1)
pg <- function(x) pt(1.5*x,df=1)

f3 <- function(x) f(x,t=1,dg=dg,pg=pg)
curve(f3,-4,4,col=2)
lines(x,dnorm(x),col=3)

enter image description here

fp3 <- function(x,p=2) x^p*f3(x)

 integrate(fp3,-Inf,Inf)  # should be just smaller than 1
0.9915525 with absolute error < 1.1e-07

 integrate(fp3,-Inf,Inf,p=4) # should be just smaller than 3
2.995066 with absolute error < 6.2e-06

 integrate(fp3,-Inf,Inf,p=4)$value/(integrate(fp2,-Inf,Inf)$value^2)
[1] 3.048917

And just to demonstrate that we have actually got a proper density:

 integrate(f3,-Inf,Inf)
1 with absolute error < 9.4e-05

Example 4: However, what happens when we change t?

Take $g$ and $G$ as the previous example, but change the threshold to $t=2$:

f4 <- function(x) f(x,t=2,dg=dg,pg=pg)
curve(f4,-4,4,col=2)
lines(x,dnorm(x),col=3)

enter image description here

fp4 <- function(x,p=2) x^p*f4(x)

 integrate(fp4,-Inf,Inf,p=4)$value/(integrate(fp2,-Inf,Inf)$value^2)
[1] 2.755231

How does this happen?

Well, it's important to know that kurtosis is (speaking slightly loosely) 1+ the squared variance about $\mu\pm\sigma$:

enter image description here

All three distributions have the same mean and variance.

The black curve is the standard normal density. The green curve shows a fairly concentrated distribution about $\mu\pm\sigma$ (that is, the variance about $\mu\pm\sigma$ is small, leading to a kurtosis that approaches toward 1, the smallest possible). The red curve shows a case where the distribution is "pushed away" from $\mu\pm\sigma$; that is the kurtosis is large.

With that in mind, if we set the threshold points far enough outside $\mu\pm\sigma$ we can push the kurtosis below 3, and still have a higher peak.

$\endgroup$
  • $\begingroup$ amazing work. Thank you. One more question, if you don't mind: is there any rule to decide where a peak ends and where tails begin? $\endgroup$ – Yal dc Dec 28 '13 at 12:20
  • 1
    $\begingroup$ Not really. If we restrict ourselves to the continuous symmetric unimodal case with finite 4th moment (since we're discussing kurtosis), in many cases I don't think it makes much sense to call anything outside $\mu \pm \sigma$ 'the peak' nor anything inside $\mu \pm \sigma$ 'the tail', but sometimes it's hard to say. e.g., consider $f(x) = (3+2a)/6 - ax^2;$ $-1<x<1, 0<a<\frac{3}{4}$; when $a$'s near $0$, there's no obvious place to start calling any of it the tail. On the other had, with the Laplace distribution, you could arguably call anything either side of the exact center the tail. $\endgroup$ – Glen_b -Reinstate Monica Dec 28 '13 at 13:23
4
$\begingroup$

Kurtosis is a rather misunderstood concept (I find L.T. De Carlo's paper "On the Meaning and Use of Kurtosis" (1997) a sensible and valuable discussion and presentation of the issues involved).

So I will take the naive view, and I will construct a density, $g_X(x)$, with "thinner middle and higher value at mode", compared to the standard normal density, but identical "tails" with the latter. I do not claim that this density exhibits "excess kurtosis".

This density will necessarily be step-wise. In order to have identical left and right "tails", its functional form for the intervals $(-\infty, -a)$ and $(a,\infty)$, where $a>0$, should be identical to the standard normal $\phi(x)$ density. In the middle interval, $(-a,a)$, it should have some other functional form, call it $h(x)$. This $h(x)$ should be symmetric around zero, and satisfy

1) $h(0) > \phi(0) = 1/\sqrt{2\pi}$ so that the value of the density at the mode will be higher than the value of the standard normal, and

2) $\phi(-a) = h(-a) = h(a) = \phi(a)$ so that $g_X(x)$ is continuous.

More over, $g_X(x)$ should integrate to unity over the domain, in order to be a proper density. So this density will be

$$g_X(x) = \begin{matrix} \phi(x) &-\infty<x\le -a\\ h(x) &-a\le x \le a\\ \phi(x) & a\le x<\infty \end{matrix}$$

subject to the previously mentioned restrictions on $h(x)$ and also, subject to

$$\int_{-\infty}^{-a}\phi(t)dt + \int_{-a}^ah(t)dt + \int_{a}^{\infty}\phi(t)dt =1$$

which is equivalent to require that the probability mass under $h(x)$ in the interval $(-a,a)$ must be equal with the probability mass under $\phi(x)$ in the same interval:

$$\int_{-a}^{-a}\left(h(t)- \phi(t)\right)dt =0 \Rightarrow \int_{0}^{a}\left(h(t)- \phi(t)\right)dt=0 $$ the last part due to the symmetry properties.

To obtain something specific, we will "try" the density of the zero-mean Laplace distribution for $h(x)$

$$h(x)= \frac 1{2b} e^{-\frac {|x|}{b}},\; b>0$$

To satisfy the various requirements set previously we must have:

For higher value at mode, $$h(0)= \frac 1{2b} > \phi(0) = \frac {1}{\sqrt{2\pi}} \Rightarrow 0<b < \sqrt{\pi/2} \qquad [1]$$

For continuity, $$h(a) = \phi(a) \Rightarrow \frac 1{2b} e^{-\frac {a}{b}} = \frac {1}{\sqrt {2\pi}}e^{-\frac 12a^2}$$ $$\Rightarrow -\ln(2b) - \frac {a}{b} = -\ln(\sqrt {2\pi}) -\frac 12a^2 \Rightarrow \frac 12a^2 - \frac {a}{b} +\ln\frac{\sqrt {\pi/2}}{b}$$

This is a quadratic in $a$. Its discriminant is $$\Delta_a = \frac 1{b^2} - 4\cdot \frac 12 \cdot\ln\frac{\sqrt {\pi/2}}{b} > 0$$

(it can be easily verified that it is always positive). More over, we keep only the positive root since $a>0$ so

$$a^* = \frac 1b + \sqrt{\Delta_a}\qquad [2]$$

Finally the requirement for the density to integrate to unity translates into

$$\int_{0}^{a^*}\frac 1{2b} e^{-\frac {|x|}{b}} dt = \int_{0}^{a^*}\phi(t)dt $$

which by straightforward integration leads to

$$1-e^{-\frac {a^*}{b}} = 2\left(\Phi(a^*) - \frac 12\right) = \operatorname{erf}(a^*/\sqrt2)\qquad [3]$$

which can be solved numerically for $b^*$, and so completely determine the density we are after.

Of course other functional forms symmetric around zero could be tried, the laplacian pdf was just for expositional purposes.

$\endgroup$
  • 1
    $\begingroup$ I found the article you've mentioned very informative. Thank you. $\endgroup$ – Yal dc Dec 28 '13 at 12:22
  • 1
    $\begingroup$ A caveat about the DeCarlo paper: The very first sentence of the abstract is mathematically incorrect. He states, “For symmetric unimodal distributions, positive kurtosis indicates heavy tails and peakedness relative to the normal distribution, whereas negative kurtosis indicates light tails and flatness.” But there are symmetric unimodal distributions with negative excess kurtosis that have infinite peaks, and there are symmetric unimodal distributions with infinite kurtosis that have perfectly flat peaks. $\endgroup$ – Peter Westfall Nov 21 '17 at 1:39
0
$\begingroup$

The kurtosis of this distribution will probably be higher than that of a normal distribution. I say probably because I am basing this on a rough drawing, and although it might be possible to prove that moving mass in this way always increases kurtosis, I am not positive about that.

Although it is true that it has the same tails as a normal distribution, this distribution will have a lower variance than the normal distribution from which it is derived. Which means that its tails will match the tails of some normal distribution, but not of a normal distribution with the same variance as it. So, the normalized tails will in fact be thicker than the tails of a normal distribution. And, although thicker tails does not automatically mean more kurtosis, in this case the normalized fourth moment will probably also be larger.

$\endgroup$
  • $\begingroup$ I agree that the variance will be lower. Unfortunately, I didn't understand how change in variance influences tails? Remember, that I did nothing to the tails. The shifted points were taken near the peak, not from the tails. Could help me to understand your point? $\endgroup$ – Yal dc Dec 27 '13 at 17:41
  • 1
    $\begingroup$ Kurtosis is defined in terms of the normalized fourth moment, where the normalization is performed by dividing by the square of the variance. Since the square of the variance goes down, kurtosis goes up. In terms of the tails, it's true that they don't change. However, since the variance has gone down, then in order to get the correct comparison, you need to compare your distribution with a normal distribution that has the same variance as yours. This other normal distribution will have thinner tails, because its variance is lower. $\endgroup$ – mpr Dec 27 '13 at 20:16
  • $\begingroup$ in that case, I agree. The question left is how did you determine what is "the correct comparison"? Is it a rule that we should use distributions with similar variance to compare their other properties? I've never met such a principle before. $\endgroup$ – Yal dc Dec 28 '13 at 7:51
  • 1
    $\begingroup$ Variance is the standard way to normalize distributions. You specifically asked about kurtosis, and as I've said, kurtosis is defined based on the normalized fourth moment, which means that if you are interested in comparing kurtosis, then yes, you should compare distributions with the same variance. $\endgroup$ – mpr Dec 28 '13 at 13:22
  • $\begingroup$ Now I understand. Indeed, any normal distribution has constant kurtosis while its variance may differ. Thank you for clarification. $\endgroup$ – Yal dc Dec 28 '13 at 13:45
0
$\begingroup$

It looks like the OP is trying to establish a connection between "peakedness" and kurtosis by keeping the tails fixed and making the distribution more "peaked." There is an effect on kurtosis here, but it is so slight that it is hardly worth a mention. Here is a theorem to support that assertion.

Theorem 1: Consider any probability distribution with finite fourth moment. Construct a new probability distribution by replacing the mass in the $[\mu - \sigma, \mu + \sigma] $ range, keeping the mass outside of $[\mu - \sigma, \mu + \sigma] $ fixed, and keeping the mean and standard deviation at $\mu, \sigma$. Then the difference between the minimum and maximum Pearson moment kurtosis values over all such replacements is $\le 0.25$.

Comment: The proof is constructive; you can actually identify the min and max kurtosis replacements in this setting. Further, 0.25 is an upper bound on the kurtosis range, depending on the distribution. For example, with a normal distribution, the range bound is 0.141, rather than 0.25.

On the other hand, there is a huge effect of tails on kurtosis, as is given by the following theorem:

Theorem 2: Consider any probability distribution with finite fourth moment. Construct a new probability distribution by replacing the mass outside the $[\mu - \sigma, \mu + \sigma] $ range, keeping the mass in $[\mu - \sigma, \mu + \sigma] $ fixed, and keeping the mean and standard deviation at $\mu, \sigma$. Then the difference between the minimum and maximum Pearson moment kurtosis values over all such replacements is unbounded; i.e., the new distribution can be chosen so that kurtosis is aribitrarily large.

Comment: These two theorems show that the effect of tails on Pearson moment kurtosis is infinite, while the effect of "peakedness" is $\le 0.25$.

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.