Why can't we reliably compare states? I came across this catchy visualization on reddit showing the percentage of adults reporting binge or heavy drinking in the US:
https://www.reddit.com/r/dataisbeautiful/comments/nve04b/oc_percentage_of_adults_reporting_binge_or_heavy/

Several users including the author warn against comparing states: "Caution should be used when comparing these estimates across state lines. The model used to create these estimates includes a state-level factor that limits comparability between neighboring counties of adjacent states". Indeed, one can see the difference between Texas and Oklahoma being at least strange.
I don't fully understand how the inclusion of a factor such as state in addition to county in a presumably additive regression model may lead to misleading interpretations. Is it a sample size problem whereby some counties have very low data? Or is it the interpretation of the coefficients?
 A: Those who posted this visualization appear not to have done the modeling that resulted in the estimates of heavy drinking displayed in the visualization.  They have simply plotted data reported here:
https://www.countyhealthrankings.org/explore-health-rankings/measures-data-sources/county-health-rankings-model/health-factors/health-behaviors/alcohol-drug-use/excessive-drinking
The source of the data is the Behavioral Risk Factor Surveillance System (also known as BRFSS).  This project is managed by the Centers for Disease Control and Prevention.  As described at the linked website:

“The Behavioral Risk Factor Surveillance System (BRFSS) is a
state-based random digit dial (RDD) telephone survey that is conducted
annually in all states, the District of Columbia, and U.S.
territories. Data obtained from the BRFSS are representative of each
state’s total non-institutionalized population over 18 years of age
and have included more than 400,000 annual respondents with landline
telephones or cellphones since 2011.”

In specific, the data for the measure “excessive drinking” used data from the 2018 Behavioral Risk Factor Surveillance System survey.
Although the BRFSS survey is large, there are counties that have only a small number of participants.  The estimate of the percentage of people who drink “excessively” by county is based on a multilevel model.  Thus, as described at the website:

” Statistical modeling is used to obtain more informed and reliable
estimates than survey data alone can provide. Our Excessive Drinking
estimates are produced from one year of survey data and are created
using complex statistical modeling. Modeling generates more stable
estimates for places with small numbers of residents or survey
responses.”

A reference to a published paper that provides technical detail on the modeling approach for making the “small area estimates” is as follows:
Zhang X, Holt JB, Lu H, Wheaton AG, Ford ES, Greenlund K, Croft JB. Multilevel regression and poststratification for small-area estimation of population health outcomes: a case study of chronic obstructive pulmonary disease prevalence using the Behavioral Risk Factor Surveillance System. American Journal of Epidemiology. 2014; 179(8):1025–1033.
The full text is available at:
https://academic.oup.com/aje/article/179/8/1025/109078
Factors in the model to estimate “excessive drinking” are age, sex, race/ethnicity (data from survey respondents) and socioeconomic status at the census tract level (data from US census).  Given the modeling strategy (and the fact that the survey sample is drawn to be representative of the population of each state), estimates for counties within a state will be close to the estimates for the state.
Others may be able to provide a more technical explanation of the model and how it affects the interpretation of differences at the borders between states.
