Why does re-scaling my density plot using counts change the y-axis so much?

Question

When I make a histogram I get the actual distribution of my samples, with the appropriate counts, but when I try making a density plot the scales go up to 800, and when I try using geom_density(aes(y = after_stat(count)) I get a y-axis scaled in the 1000s. I assume this is the expected behaviour of ggplot2, but I just don't understand the theory behind this.

Here are the histograms:

Here are the normal density plots:

And here's the density plots with after_stat(count):

Edit:

The x axis represents percentages I calculated manually to aggregate the lipid concentrations I was given. So it's something like "percentage of total lipid concentration", that's why the x axes don't always start at 0. Was I wrong to represent percentages in histograms/density plots?

Answer 1

The bottom line (here at the top) is that the vertical scale in the second series of plots is probability density, which not only is not the count or frequency in any bin, it is not probability either. It is the probability per unit, where the unit (of measurement) is the unit of whatever is plotted on the horizontal axis.

The rule is that the probability density (probability per unit) adds (strictly, integrates) to 1 over the total support. To see this, think of a uniform (rectangular, flat) distribution on [0, 1]. The range is 1, the probability density is constant at 1, and

total probability = average density over the range $\times$ the range

OR

1 = 1 (this average is easy, as density is constant) $\times$ 1.

Now halve the variable so that the distribution is on [0, 0.5]. The total probability remains the same and so the average density must double to 2. Or double it, and the converse is observed.

Checking out your second series of plots underlines that as the horizontal range increases numerically, even though the specific units may well vary from plot to plot, so also the typical densities decrease. This inverse relationship arises from the fact that total probability is constant at 1.

The top left distribution has a range of about 0.007, so the average density must be about 1/ 0.007 or about 140, which does not quite match the numbers on the probability density axis. At the same time, it seems that the plots are all truncated, so the set-up is presumably complicated by that at least.

In principle, this is nothing whatever to do with your use of R or ggplot2. It is standard statistically.

Important detail: What the third series of plots shows is not so clear to me. Perhaps it is frequency density.

One way to check what any routine does is to feed it a very simple dataset for which you can work out in advance what the results should be. E.g. 100 values of integers 0 to 9 with equal frequencies of 10.

Answer 2

The difference between the two figures is that one displays the probability density and the other the count density.

The total probability is always one (by definition). But the total counts can be different for the three groups. This makes that the relative scales of the two different density plots is different.

For example: in the c19 steroids measurements you have only 3 cases from the control group and 13 cases from the hyper group. (I am guessing those numbers based on your histogram below)

So the number/count density for the hyper group is much higher than the number/count density for the control group. (see below the green curves/area which is much higher than the red curve/area)

When you use probability density, then the scale for each distribution is seperately normalized, such that the total area under the curve equals to 1. (It is the integral that needs to sum up to 1, below this is roughly estimated with two rectangles that seem to have a similar area as the area under the curves)