I have data for US ~3100 counties where the variable is a mean score based on a sample. However, for many small counties, the sample size is quite small (like 5), and so these mean values fluctuate a lot and result in extreme values. I need to use this variable as an outcome in a regression and for plotting, so the fluctuations are a problem for me (I use weights in regression but for plotting purposes, the data has to be adjusted itself).
One seemingly common approach in spatial statistics is to use the empirical Bayes approach to move the values towards the global mean. However, the examples I can find of this approach all refer to count data, whereas my data are continuous.
Example of spatially informed empirical Bayes adjustment from link 2.
Here's a simple fictive example in R (without the spatial component).
#simulate population sizes, true means, and observed means library(tidyverse) library(truncnorm) #sim data d = tibble( n = runif(1e4, min = 2, max = 100) %>% round(), true_mean = rnorm(1e4, mean = 100, sd = 15), observed_mean = NA ) #sample the means d$observed_mean = map2_dbl(d$n, d$true_mean, ~mean(rnorm(.x, mean = .y, sd = 15))) #plot ggplot(d, aes(true_mean, observed_mean, color = n)) + geom_point(alpha = .5)
which gives this plot:
We see that the smaller samples tend to produce outlying values, which one would need to shrink somewhat towards the mean.
My questions are:
- Is there a simple function for adjusting these values towards the mean in the same way as in the empirical Bayes examples?
- Is there a spatially informative version of this approach as in the first link?
I am looking for an R solution.