1
$\begingroup$

I am preparing to do a regression model (probably Poisson) where the goal variable is count data (number of times a patient received a blood transfusion). I will have to do variable selection out of many available independent variables, and want to explore the data thoroughly before doing the actual regression.

I am accustomed to starting with histograms and scatterplots. But of course, the scatterplot shape is very different from a scatterplot with a continuous variable, consisting mostly of flat lines below the outliers.

scatterplot

For what it's worth, I did a univariate linear regression, and for both counts, it was highly significant, in the 10-7 and 10-6 region respectively. So if I were going off that information, I would expect to see at least some relationship visually, even though it won't be scattered around an actual straight line. But I don't.

How do I interpret this scatterplot? What better visualization are available for the relationship of the explanatory variable to count data?

$\endgroup$
2
  • 5
    $\begingroup$ These plots can be deceptive due to the extensive overplotting of points. When that occurs, a simple, quick recourse is to jitter the values. Even a little bit along the vertical axis ($\pm 1/4$ or so in each direction) will give a clearer indication of the distributions without ruining your ability to read off the actual counts. Using semi-transparent point symbols also helps a lot. $\endgroup$
    – whuber
    Commented May 16, 2022 at 21:02
  • $\begingroup$ @Rumi "I will have to do variable selection out of many available independent variables, and want to explore the data thoroughly before doing the actual regression." --- be careful of using inference on the same data you use to identify models / select variables. If you must do both, consider sample splitting to avoid the issue of the impact that the data-based choice of analysis has on properties of the inference. $\endgroup$
    – Glen_b
    Commented May 18, 2022 at 1:04

2 Answers 2

4
$\begingroup$

I'm a fan of exploratory data analysis, but you should be cautious of it and the intended variable selection. If you make any decisions about which variables to use or how to model the data based on what you see in the data, the resulting model will be biased and any inferences (assuming that's of interest) would be invalid. One possibility is to randomly partition the data into two subsets, explore one, and pursue your chosen model on the other. If you're not interested in inference, you could use a LASSO model to get variable selection.

The shapes you see in these plots are common for count data, which are typically highly skewed. Furthermore, it isn't at all clear that there are any actual 'outliers'. Individual points that lie beyond the bulk of the data are normal for count data like these. These plots do look like they show potential relationships.

To help you view them, I might suggest trying several things:

  1. Experiment with some transformations. I wouldn't be surprised if log transformations are appropriate for both X and Y. (Remember, though, that a Poisson GLM uses a transformation of the conditional mean, not a transformation of the data, so this won't be quite the same.)
  2. The resulting plots will still have stripes. If the skews and variances have stabilized, you could try adding a small amount of jitter to nudge the data apart.
  3. Likewise, making the data semi-transparent may similarly help.
  4. Lastly, it may be beneficial to plot a LOWESS line over the scatterplot to help see the relationship.
$\endgroup$
3
$\begingroup$

In addition to the other great answers (use vertical jitter and transparency aka alpha to deal with overplotting!), you could consider 2d binning. It's a 2d analogue to a histogram.

In your case, only the $y$-axis seems to be discrete. I imagine you'd want to choose the vertical bins to be centered on the integers, and try several different horizontal bin widths to see which ones give a clearer visual representation of any trends in the data.

Since you seem to be using ggplot2, you could easily add a layer with geom_bin_2d() or geom_hex(). The former makes rectangular bins, while the latter's hexagonal bins are meant to avoid visual artifacts than might come from a regular grid. In your case, a regular grid is likely to be more appropriate since it would honestly show that only $y$ is discrete, but you could try it both ways.
It may also help to overlay the (transparent) points on top of the 2d bins, to distinguish bins with 1 observation from bins with just a few observations.

More generally, if both $x$ and $y$ are discrete, then also consider an approach like geom_count() which uses point size to represent the number of observations at each $(x,y)$ location.

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.