7
$\begingroup$

I have a data set with about 15,000 labeled observations of a single continuous value. What is the best way to plot this type of data? I'm playing around with various histograms and density plots, but I can't seem to figure out the best way to plot this data set. Any suggestions?

My data looks like this: 

 label     value  
-------   -------  
  foo       1.2  
  bar       6.2  
  baz       0.2  
  qux       4.7  
  ...       ...

This data set contains 15,000 values, each with a unique label.

I am looking as to how best to create a visualization of the distribution of the data and see outliers. Here are two candidate plots I've generated. Both simplified the data more than I would like. Are there any additional ways I could plot the data and somehow integrate the labels into this plot?

density plot boxplot

Improve this question
$\endgroup$
11
  • $\begingroup$ Could you clarify what you mean by a "discrete" variable with a single "continuous" value? This seems self-contradictory. $\endgroup$
    – whuber ♦
    Jan 9, 2012 at 18:16
  • $\begingroup$ I have variables such as x, y, z, etc. Each variable has a continuous values such as x=0.2, y=1.2, z=2.9. $\endgroup$
    – dr.bunsen
    Jan 9, 2012 at 18:18
  • $\begingroup$ Normally we think of such data as arrayed in a table in which variables (or "attributes" or "fields") are given by columns and cases (or "observations") are given by rows. You're saying you have a table with 15,000 columns. How many rows are in it? $\endgroup$
    – whuber ♦
    Jan 9, 2012 at 18:21
  • 2
    $\begingroup$ The edit makes the question much clearer, thanks. Now that the format and general nature of your data are revealed, could you tell us what you would like the visualization to show? (Obviously, unless there is some structure to the labels, no visualization will be capable of displaying 15000 distinct labels in any clear or meaningful way.) It would help to say something about what the values mean. Are there natural limits to their ranges? What is their actual range? Are they perhaps proportions, or are they direct measurements of something? $\endgroup$
    – whuber ♦
    Jan 9, 2012 at 18:47
  • 2
    $\begingroup$ Now we have a clear, answerable question! (+1). I look forward to reading the solutions people have to offer. (Automatic labeling of plots is notoriously tricky and manual labeling is a pain, so there is a real problem to be addressed here.) $\endgroup$
    – whuber ♦
    Jan 9, 2012 at 19:07

2 Answers 2

Reset to default
6
$\begingroup$

A simple idea that satisfies both seeing the entire distribution and one that potentially allows space to place the labels is to use a jittered, one-dimensional dot plot.

enter image description here

In this example I have just arbitrarily chosen points with a value over 25 to label, as well as made the points semi-transparent (even with jittering there is still a substantial amount of over-plotting).

This isn't perfect (I had to arbitrarily choose what points where deemed to be "outliers" and hence which ones would get the label). Also in the software I produced this in (SPSS) I can't restrict the jittering only to the Y dimension of the graph, so many points in this instance fall outside the logical ranges (this is random data generated using a log-normal distribution, so all values are positive).

Although I hope this is a useful demonstration for plotting, given the look of your examples you would surely want to consider transforming the data. I'm sure other members of the site can produce some examples in ggplot2 for demonstration if you need help producing a similar graph in R as well (I look forward to other suggestions as well!)

Improve this answer
$\endgroup$
1
  • $\begingroup$ +1 Nice idea for using the second dimension to create space for the labels. $\endgroup$
    – whuber ♦
    Jan 9, 2012 at 20:58
6
$\begingroup$

Expanding on the idea from @Andy, I would suggest this alternative which incorporates the two original plots included in the questions:labeled outliers

I created this plot in R with the following code:

data <- data.frame( Id = paste('case',1:5000), x = exp(rnorm(5000)))

outlier.cut = 10

outliers <- data[ data$x > outlier.cut, ]
outliers <- outliers[ order(outliers$x), ]

png(filename = "labeled_outliers.png", width = 600, height = 600)

plot(density(data$x), main = 'Labeled Outliers')
rug( data$x, side = 1)
rug( data$x, side = 3)

unit <- (par('usr')[4]-par('usr')[3]) / ( dim(outliers)[1] + 5 )
outliers$y <- ( c(1:dim(outliers)[1]) * unit)

text(outliers$x, y = outliers$y, label = outliers$Id, cex = .65, srt = 0)

dev.off()
Improve this answer
$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge that you have read and understand our privacy policy and code of conduct.

Not the answer you're looking for? Browse other questions tagged or ask your own question.