2
$\begingroup$

I have data with around 25,000 rows myData with column attr having values from 0 -> 45,600. I am not sure how to make a simplified or reproducible data...

Anyway, I am plotting the density of attr like below, and I also find the attr value where density is maximum:

library(ggplot)
max <- which.max(density(myData$attr)$y)
density(myData$attr)$x[max] # This is the x-intercept of max point

ggplot(myData, aes(x=attr))+ 
  geom_density(color="darkblue", fill="lightblue")+
  geom_vline(xintercept = density(myData$attr)$x[max])+
  xlab("attr")

Here is the plot I have got with the x-intercept at maximum point: enter image description here

Since the data is skewed, I then attempted to draw x-axis in log scale by adding scale_x_log10() to the ggplot, here is the new graph: enter image description here

My questions now are:

1. Why does it have 2 maximum points now? Why is my x-intercept no longer at the maximum point(s)?

2. How do I find the intercepts for the 2 new maximum points?

$\endgroup$
1
  • $\begingroup$ Is zero an observed value in your data? $\endgroup$
    – Nick Cox
    Nov 10, 2019 at 8:12

1 Answer 1

5
$\begingroup$
  1. Be very careful about just transforming the x-axis; that doesn't represent the density on the log scale of the variable because it omits the Jacobian of the transformation. Just transforming the x-axis works for the cdf, but not for the corresponding pdf (which you can see by differentiating the cdf).

  2. When you transform a distribution with a nonlinear transformation, you change the relative local density (ratios of probabilities of being within small intervals $[x,x+dx)$ are changed); modes are not equivariant to monotonic transformation.

    Note, for example, that the mode of a normal is at the median (which is also the mean), but the mode of a lognormal is below the median, which in turn is below the mean.

  3. KDEs add some wrinkles to this

    a. transformation changes relative distance between points

    b. smoothing (via kernel density estimates) a transformed density means combining kernels from nearby points; consequently, the contributions of points is altered because with conventional KDEs the bandwith is constant on the new scale, not transformed from the old scale (i.e. if "s" is the operation of computing a KDE by some given typical algorithm, then $s(t(X)) \neq t(s(x))$).

    (Note also that transforming the x-axis on the kernel density estimate can introduce odd artifacts relative to just transforming the axis on the distribution that the density is from. Beware that sometimes you can be looking at an attribute of a transformed kernel rather than a transformed density. How strong this effect is depends on various factors. Note, for example, that if you took a density on the positive half-line with considerable density near 0, and computed a KDE and transformed the axis, you'll have positive density at and below 0 and the effect of taking say a log of that may not be what you'd hope for.)

Long story short, for several reasons at once, you should expect peaks to shift, for the number of peaks to potentially change, and for transforming just the x-axis to not do quite what you might expect it to do.

$\endgroup$
2
  • $\begingroup$ Thank you for your response. So in my case, it's better to stick with the original graph? Is there any way to better show the data? since it's too skewed. $\endgroup$
    – hydradon
    Nov 10, 2019 at 8:07
  • $\begingroup$ Transformation is often okay, as long as transforming the variable is what you want. Indeed, quite often a well chosen transformation is productive, if the transformed scale is meaningful. $\endgroup$
    – Glen_b
    Nov 10, 2019 at 9:45

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.