10
$\begingroup$

I have some data I need to visualize and am not sure how best to do so. I have some set of base items $Q = \{ q_1, \cdots, q_n \}$ with respective frequencies $F = \{f_1, \cdots, f_n \}$ and outcomes $O \in \{0,1\}^n$. Now I need to plot how well my method "finds" (i.e., a 1-outcome) the low frequency items. I initially just had an x-axis of frequency and a y axis of 0-1 with point-plots, but it looked horrible (especially when comparing data from two methods). That is, each item $q \in Q$ is has an outcome (0/1) and is ordered by its frequency.

Here is an example with a single method's results:

enter image description here

My next idea was to divide the data into intervals and compute a local sensitivity over the intervals, but the problem with that idea is the frequency distribution is not necessarily uniform. So how should I best pick the intervals?

Does anyone know of a better/more useful way to visualize these sort of data to portray the effectiveness of finding rare (i.e., very low-frequency) items?

EDIT: To be more concrete, I am showcasing the ability of some method to reconstruct biological sequences of a certain population. For validation using simulated data, I need to show the ability to reconstruct variants regardless of its abundance (frequency). So in this case I am visualizing the missed and found items, ordered by their frequency. This plot will not include reconstructed variants that are not in $Q$.

$\endgroup$
  • 1
    $\begingroup$ I don't completely understand. Are the "outcomes" finding something? What are the "rare items"? $\endgroup$ – Peter Flom - Reinstate Monica Dec 8 '12 at 22:12
  • 1
    $\begingroup$ IMO you should include the graph you said looks horrible - it will give everyone a better idea of the data you are attempting to display. $\endgroup$ – Andy W Dec 8 '12 at 22:12
  • $\begingroup$ @PeterFlom, I've edited to make it clearer. The 0-1 outcomes for each item indicate "not-found" and "found". A rare item is simple a very low frequency item. $\endgroup$ – Nicholas Mancuso Dec 8 '12 at 22:19
  • $\begingroup$ @AndyW, edited to include image. Given the values on the y-axis don't really reflect the concept of found and not found, but at least for conveying what I want to present (for this questions' purposes), you get the idea... $\endgroup$ – Nicholas Mancuso Dec 8 '12 at 22:20
  • 1
    $\begingroup$ OK, it looks like you tried a scatterplot on data where the y value can only be 0 or 1. Is that right? And you want to compare these sorts of plots across multiple methods on the same points? But can each method be right or wrong in one way or two ways? That is, each point either is or is not (whatever). So a method could say a point is (whatever) or isn't (whatever) and either choice could be right or wrong? $\endgroup$ – Peter Flom - Reinstate Monica Dec 8 '12 at 22:29
10
$\begingroup$

What I have done in the past is basically what you've done with the addition of a loess. Depending on the density of points, I would use translucent points (alpha), as shown below, and/or pipe symbols ("|") to minimize overlap.

library(ggplot2) # plotting package for R

N=100
data=data.frame(Q=seq(N), Freq=runif(N,0,1), Success=sample(seq(0,1), 
size=N, replace=TRUE))

ggplot(data, aes(x=Freq, y=Success))+geom_point(size=2, alpha=0.4)+
  stat_smooth(method="loess", colour="blue", size=1.5)+
  xlab("Frequency")+
  ylab("Probability of Detection")+
  theme_bw()

enter image description here

(I don't think the error bars should widen on the edges here, but there isn't an easy way I know of to do that with ggplot's internal stat_smooth function. If you used this method for reals in R, we could do it by estimating the loess and its error bar before plotting.)

(Edit: And plus-ones for comments from Andy W. about trying vertical jitter if the density of the data makes it useful and from Mimshot about proper confidence intervals.)

$\endgroup$
  • 3
    $\begingroup$ +1 - I would also suggest utilizing jitter for the dots (in addition to transparency). In this example I would replace geom_point(size=2, alpha=0.4) with geom_jitter(size=2, alpha=0.4, position = position_jitter(height = .02)). $\endgroup$ – Andy W Dec 9 '12 at 15:45
  • 3
    $\begingroup$ +1 but you should use confidence bounds from the inverse of the binomial distribution rather than you're implied Gaussian noise. $\endgroup$ – Mimshot Dec 9 '12 at 17:27
  • $\begingroup$ @Mimshot Can you show how to compute the confidence intervals correctly? $\endgroup$ – bee guy Apr 10 '18 at 7:50
  • 1
    $\begingroup$ @Mimshot do you know of a way within ggplot2 to supply the correct CIs? I've got a plot with CIs outside of [0,1] which clearly are coming from the wrong calculation $\endgroup$ – MichaelChirico Jun 18 '18 at 6:15
  • $\begingroup$ This is a good answer (+1), but the range of the plot (and CIs) should definitely be restricted onto the interval $[0,1]$. That would improve the look of the plot and also respect the support of the allowable probability value. $\endgroup$ – Ben - Reinstate Monica Feb 26 '19 at 8:47
2
$\begingroup$

Also consider which scales are most appropriate for your use case. Say you're doing visual inspection for the purposes of modeling in logistic regression and want to visualize a continuous predictor to determine if you need to add a spline or polynomial term to your model. In this case, you may want a scale in log-odds rather than probability/proportion.

The function at the gist below uses some limited heuristics to split the continuous predictor into bins, calculate the mean proportion, convert to log-odds, then plot geom_smooth over these aggregate points.

Example of what this chart looks like if a covariate has a quadratic relationship (+ noise) with the log-odds of a binary target:

devtools::source_gist("https://gist.github.com/brshallo/3ccb8e12a3519b05ec41ca93500aa4b3")

# simulated dataset with quadratic relationship between x and y
set.seed(12)
samp_size <- 1000
simulated_df <- tibble(x = rlogis(samp_size), 
                       y_odds = 0.2*x^2,
                       y_probs = exp(y_odds)/(1 + exp(y_odds))) %>% 
  mutate(y = rbinom(samp_size, 1, prob = y_probs)) 

# looking at on balanced dataset
simulated_df_balanced <- simulated_df %>% 
  group_by(y) %>% 
  sample_n(table(simulated_df$y) %>% min())


ggplot_continuous_binary(df = simulated_df,
                         covariate = x, 
                         response = y,
                         snip_scales = TRUE)
#> [1] "bin size: 18"
#> `geom_smooth()` using method = 'loess' and formula 'y ~ x'

Created on 2019-02-06 by the reprex package (v0.2.1)

For comparison, here is what that quadratic relationship would look like if you just plotted the 1's/0's and added a geom_smooth:

simulated_df %>% 
  ggplot(aes(x, y))+
  geom_smooth()+
  geom_jitter(height = 0.01, width = 0)+
  coord_cartesian(ylim = c(0, 1), xlim = c(-3.76, 3.59))
# set xlim to be generally consistent with prior chart
#> `geom_smooth()` using method = 'gam' and formula 'y ~ s(x, bs = "cs")'

Created on 2019-02-25 by the reprex package (v0.2.1)

Relationship to logit is less clear and using geom_smooth has some problems.

$\endgroup$
0
$\begingroup$

I agree that posting just a few lines of sample data would go a long way. If I understand the question, I think it would be simplest to plot the frequency by the proportion found.

First I will generate some sample data in R; please correct me if I haven't understood you correctly.

# Create some sample data
data=data.frame(Q=1:20,F=seq(5,100,by=5))
set.seed(1)
data$found<-round(sapply(data$F,function(x) runif(1,1,x)))
data$prop<-data$found/data$F
# Looks like:
Q   F found      prop
1   1   5     2 0.4000000
2   2  10     4 0.4000000
3   3  15     9 0.6000000
4   4  20    18 0.9000000
5   5  25     6 0.2400000
6   6  30    27 0.9000000
7   7  35    33 0.9428571
8   8  40    27 0.6750000
9   9  45    29 0.6444444
10 10  50     4 0.0800000
11 11  55    12 0.2181818
12 12  60    11 0.1833333
13 13  65    45 0.6923077
14 14  70    28 0.4000000
15 15  75    58 0.7733333
16 16  80    40 0.5000000
17 17  85    61 0.7176471
18 18  90    89 0.9888889
19 19  95    37 0.3894737
20 20 100    78 0.7800000

And now simply plot frequency (F) by proportion:

# Plot frequency by proportion found.
plot(data$F,data$prop,xlab='Frequency',ylab='Proportion Found',type='l',col='red',lwd=2)

enter image description here

$\endgroup$
  • 4
    $\begingroup$ That plot is horrible! Some smoothing, as in earlier answers, are necessary. $\endgroup$ – kjetil b halvorsen Dec 9 '12 at 17:14

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.