2
$\begingroup$

I have some code (shown below) for sampling from and then estimating a normal mixture distribution in one dimension. When I plot the estimate (blue) against the true distribution (black) I get images of the following form: enter image description here

My issue is that the estimated distribution seems to be ALWAYS more spread out with peaks that are lower than the peaks of the true distribution. I would have expected that since the estimated distribution is random, that sometimes it would have higher peaks and sometimes it would have lower peaks than the true distribution. But this is not the case as you can see by running the code.

If I increasing the sampling from N=1e2 to say N=1e4, the peaks of the estimated distribution get closer to the those of the true distribution, but they are still always lower.

So what's the cause of this phenomenon, why is the estimated distribution more spread out with peaks that are lower than those of the true distribution?

library(distr)

N = 1e2
means = c(4,20)
sds = c(3,3)
w_1 = 0.4
ws = c(w_1,1-w_1)

myMix <- UnivarMixingDistribution(Norm(mean=means[1], sd=sds[1]), 
                                  Norm(mean=means[2], sd=sds[1]),
                                  mixCoeff=ws)

# Reference distribution
dmyMix <- d(myMix)
grid_Min = -9
grid_Max = 35
grid_N = 5e2

# Sample from distribution
rmyMix <- r(myMix)
Y <- rmyMix(N)
f_hat_density <- density(Y)

x_Min = -12
x_Max = 37
x_N = 512

f_hat_density <- density(Y, from=x_Min, to=x_Max, n=x_N)

x = f_hat_density$x
f_approx = f_hat_density$y
f_exact = dmyMix(x)

# Plot
plot(x,f_exact, type="l")
lines(x,f_approx, col='blue',lty=2)
$\endgroup$
2
  • 1
    $\begingroup$ I would guess this is caused by the density() function smearing the estimated pdf sideways and so bringing down the height. This could explain why larger sample sizes reduce the effect $\endgroup$
    – Henry
    Commented Feb 2, 2021 at 15:35
  • 1
    $\begingroup$ density is smoothing over a window, so it tends to average out the peaks and valleys. It's sort of like using a moving average. $\endgroup$
    – John L
    Commented Feb 2, 2021 at 15:39

1 Answer 1

1
$\begingroup$

It is a function of the bandwidth. Larger bandwidth smooths out and reduces the heights of the peaks, smaller bandwidth does the reverse. Try using

f_hat_density <- density(Y, adjust=.1, from=x_Min, to=x_Max, n=x_N)

to choose a bandwidth one tenth the size of the default. This will give you higher peaks in your estimated density.

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.