How do you visualize binary outcomes versus a continuous predictor?

Question

I have some data I need to visualize and am not sure how best to do so. I have some set of base items $Q = \{ q_1, \cdots, q_n \}$ with respective frequencies $F = \{f_1, \cdots, f_n \}$ and outcomes $O \in \{0,1\}^n$. Now I need to plot how well my method "finds" (i.e., a 1-outcome) the low frequency items. I initially just had an x-axis of frequency and a y axis of 0-1 with point-plots, but it looked horrible (especially when comparing data from two methods). That is, each item $q \in Q$ is has an outcome (0/1) and is ordered by its frequency.

Here is an example with a single method's results:

My next idea was to divide the data into intervals and compute a local sensitivity over the intervals, but the problem with that idea is the frequency distribution is not necessarily uniform. So how should I best pick the intervals?

Does anyone know of a better/more useful way to visualize these sort of data to portray the effectiveness of finding rare (i.e., very low-frequency) items?

EDIT: To be more concrete, I am showcasing the ability of some method to reconstruct biological sequences of a certain population. For validation using simulated data, I need to show the ability to reconstruct variants regardless of its abundance (frequency). So in this case I am visualizing the missed and found items, ordered by their frequency. This plot will not include reconstructed variants that are not in $Q$.

Answer 1

What I have done in the past is basically what you've done with the addition of a loess. Depending on the density of points, I would use translucent points (alpha), as shown below, and/or pipe symbols ("|") to minimize overlap.

library(ggplot2) # plotting package for R

N=100
data=data.frame(Q=seq(N), Freq=runif(N,0,1), Success=sample(seq(0,1), 
size=N, replace=TRUE))

ggplot(data, aes(x=Freq, y=Success))+geom_point(size=2, alpha=0.4)+
  stat_smooth(method="loess", colour="blue", size=1.5)+
  xlab("Frequency")+
  ylab("Probability of Detection")+
  theme_bw()

(I don't think the error bars should widen on the edges here, but there isn't an easy way I know of to do that with ggplot's internal stat_smooth function. If you used this method for reals in R, we could do it by estimating the loess and its error bar before plotting.)

(Edit: And plus-ones for comments from Andy W. about trying vertical jitter if the density of the data makes it useful and from Mimshot about proper confidence intervals.)

Answer 2

Also consider which scales are most appropriate for your use case. Say you're doing visual inspection for the purposes of modeling in logistic regression and want to visualize a continuous predictor to determine if you need to add a spline or polynomial term to your model. In this case, you may want a scale in log-odds rather than probability/proportion.

The function at the gist below uses some limited heuristics to split the continuous predictor into bins, calculate the mean proportion, convert to log-odds, then plot geom_smooth over these aggregate points.

Example of what this chart looks like if a covariate has a quadratic relationship (+ noise) with the log-odds of a binary target:

devtools::source_gist("https://gist.github.com/brshallo/3ccb8e12a3519b05ec41ca93500aa4b3")

# simulated dataset with quadratic relationship between x and y
set.seed(12)
samp_size <- 1000
simulated_df <- tibble(x = rlogis(samp_size), 
                       y_odds = 0.2*x^2,
                       y_probs = exp(y_odds)/(1 + exp(y_odds))) %>% 
  mutate(y = rbinom(samp_size, 1, prob = y_probs)) 

# looking at on balanced dataset
simulated_df_balanced <- simulated_df %>% 
  group_by(y) %>% 
  sample_n(table(simulated_df$y) %>% min())


ggplot_continuous_binary(df = simulated_df,
                         covariate = x, 
                         response = y,
                         snip_scales = TRUE)
#> [1] "bin size: 18"
#> `geom_smooth()` using method = 'loess' and formula 'y ~ x'

^{Created on 2019-02-06 by the reprex package (v0.2.1)}

For comparison, here is what that quadratic relationship would look like if you just plotted the 1's/0's and added a geom_smooth:

simulated_df %>% 
  ggplot(aes(x, y))+
  geom_smooth()+
  geom_jitter(height = 0.01, width = 0)+
  coord_cartesian(ylim = c(0, 1), xlim = c(-3.76, 3.59))
# set xlim to be generally consistent with prior chart
#> `geom_smooth()` using method = 'gam' and formula 'y ~ s(x, bs = "cs")'

^{Created on 2019-02-25 by the reprex package (v0.2.1)}

Relationship to logit is less clear and using geom_smooth has some problems.

Answer 3

If you have so many points that they overlap and jittering is insufficient, you can add histograms for both levels of the binary variables (so one will be upside down). Here's an example combined with a logistic regression.

The idea and R code (popbio::logi.hist.plot) come from

Rot, M. D. L. C. (2005). Improving the presentation of results of logistic regression with R. Bulletin of the Ecological Society of America, 86(1), 41-48.

https://esajournals.onlinelibrary.wiley.com/doi/10.1890/0012-9623%282005%2986%5B41%3AITPORO%5D2.0.CO%3B2

Answer 4

The ggridges package offers more creative ways to avoid overplotting those ones and zeros. Modifying @MattBagg's example. Not optimal for this dataset, but you'll get the point.

library(ggplot2) 
library(ggridges)

N=100
data=data.frame(Q=seq(N), Freq=runif(N,0,1), 
                Success=sample(seq(0,1), size=N, replace=TRUE))

ggplot() + 
  ggridges::geom_density_ridges(data = data, aes(x = Freq, y = Success, group = Success), scale = 0.2) +
  stat_smooth(data = data, aes(x = Freq, y = Success), size=1.5) +
  coord_cartesian(ylim = c(0, 1.25), xlim = c(0, 1), expand = FALSE) +
  scale_y_continuous(breaks = c(0, 0.5, 1)) +
  labs(x = "Frequency", y = "Probability of Detection") +
  theme_bw()
#> Picking joint bandwidth of 0.123
#> `geom_smooth()` using method = 'loess' and formula 'y ~ x'

^{Created on 2021-06-30 by the reprex package (v2.0.0)}

Answer 5

I agree that posting just a few lines of sample data would go a long way. If I understand the question, I think it would be simplest to plot the frequency by the proportion found.

First I will generate some sample data in R; please correct me if I haven't understood you correctly.

# Create some sample data
data=data.frame(Q=1:20,F=seq(5,100,by=5))
set.seed(1)
data$found<-round(sapply(data$F,function(x) runif(1,1,x)))
data$prop<-data$found/data$F
# Looks like:
Q   F found      prop
1   1   5     2 0.4000000
2   2  10     4 0.4000000
3   3  15     9 0.6000000
4   4  20    18 0.9000000
5   5  25     6 0.2400000
6   6  30    27 0.9000000
7   7  35    33 0.9428571
8   8  40    27 0.6750000
9   9  45    29 0.6444444
10 10  50     4 0.0800000
11 11  55    12 0.2181818
12 12  60    11 0.1833333
13 13  65    45 0.6923077
14 14  70    28 0.4000000
15 15  75    58 0.7733333
16 16  80    40 0.5000000
17 17  85    61 0.7176471
18 18  90    89 0.9888889
19 19  95    37 0.3894737
20 20 100    78 0.7800000

And now simply plot frequency (F) by proportion:

# Plot frequency by proportion found.
plot(data$F,data$prop,xlab='Frequency',ylab='Proportion Found',type='l',col='red',lwd=2)