9
$\begingroup$

The second peak of the density plot is large in this example. Why does the rug representation of the data--which seems to show few high values--not appear to match the much higher density estimated there?

How can one make rug plots less misleading?

enter image description here

Here's its R code:

plot(density(Boston$tax))
rug(Boston$tax, col=2, lwd=3.5)
$\endgroup$
4
  • 2
    $\begingroup$ You are using subjective terms like "so high" and "inadequate"...why is the second peak "too high" and why is the rug function not acceptable? What is it doing that you don't like. $\endgroup$
    – user31668
    Commented Feb 12, 2014 at 4:40
  • $\begingroup$ This question appears to be relying on unjustified premises (at least in general). Are these questions someone else has set for you to do? $\endgroup$
    – Glen_b
    Commented Feb 12, 2014 at 6:49
  • 2
    $\begingroup$ How many data points are above the level of 600? $\endgroup$
    – Michael M
    Commented Feb 12, 2014 at 7:25
  • 5
    $\begingroup$ The "problem" with the rug plot is just that one tick and several ticks in the same place look the same. You could use a spike for each distinct value whose length was proportional to number of values, i.e. a histogram with bins the distinct values. $\endgroup$
    – Nick Cox
    Commented Feb 12, 2014 at 11:05

2 Answers 2

11
$\begingroup$

From the R package MASS, of the $506$ total observations in Boston, $369$ have a value for tax below 470 and $137$ have a value for tax above 665. In fact 666 is by far the most common value in the data set, appearing $132$ times.

So if the area of the density plot to the left is about twice the area to the right, then that could reasonably be taken as representing the distribution. Visual inspection suggests this might be what is happening.

A more accurate representation would have the right peak much higher and narrower, and this could be achieved by adjusting the parameters.

Added for comments:

For example with a much narrower bandwidth for the density function and some manual jitter:

library(MASS)
plot(density(Boston$tax, bw=5)) 
rug(Boston$tax + rnorm(length(Boston$tax), sd=5), col=2, lwd=3.5)

you would get something like this

Bostontaxdensity

$\endgroup$
5
  • 1
    $\begingroup$ You have hit upon the reason for the discrepancy. But upon adjusting the parameters of the density estimate, the area of the right peak should not change and so there will still be a strong visual discrepancy between what the rug plot displays and the density plot displays. Effective solutions will--as the O.P. suggests and as @Nick Cox hints in a comment--modify the rug plot rather than the density plot. $\endgroup$
    – whuber
    Commented Feb 12, 2014 at 16:50
  • 3
    $\begingroup$ Yeah, I used jitter for modifying the rug. $\endgroup$
    – Mona Jalal
    Commented Feb 12, 2014 at 17:09
  • 1
    $\begingroup$ @MonaJalal could you post what the chart looks like after you made your changes? I'd be interested in seeing it. $\endgroup$ Commented Apr 4, 2016 at 17:34
  • 2
    $\begingroup$ @RickHenderson: I added a chart with a narrower bandwidth and manual jitter $\endgroup$
    – Henry
    Commented Apr 4, 2016 at 19:18
  • $\begingroup$ Jitter on a rug plot defeats the very purpose of indicating that these values are the distinct values that occur in the sample. NB distinct, not unique, which is the pivot of the question. $\endgroup$
    – Nick Cox
    Commented Aug 3, 2022 at 6:48
16
$\begingroup$

The way to make a rug plot less misleading is often to use something different instead. Rug plots necessarily are relatively good at showing distinct values and very poor at indicating their relative frequency.

Here is a spike representation of the frequency distribution of the data used in the original post. The principle is that often used to show discrete distributions, a spike proportional in height to the frequency of each distinct value. The Boston data as available and documented here were read into Stata and the spikeplot command used. Something similar should be trivial in all good statistical software. If you like, this is a hybrid of a histogram and a rug plot, although historically graphs of this kind may long predate rug plots.

enter image description here

Stata documentation for spikeplot, including, more to the point of this question, further examples of such graphs, is accessible here

EDIT 3 August 2022 See also this paper for examples of marginal spike histograms as inspired by Harrell, F.E. 2001/2015. Regression Modeling Strategies. Springer.

The Boston housing data are often used as an example. Many researchers follow in the path of the original authors, who did not show any plots of the raw data in their paper. Here are four of the more unusual distributions for predictors in that dataset, as shown by quantile plots, plots of the ordered values versus an estimate of cumulative probability. The vertical axis labels are minimum, lower quartile, median, upper quartile and maximum, which in some instances coincide. A quantile plot is good for showing marked spikes and gaps in the data. Naturally an unusual (e.g. spiky) distribution doesn't stop a variable being useful as a predictor, as exemplified also by indicator or dummy variables, but it is a good idea to look at your data before modelling.
In some problems, there may be real doubts about data quality, or at least puzzles about definitions and measurements that need to inform the interpretation.

enter image description here

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.