# Can a significance test be done on two variables which are 3,140 county averages? (Significantly different sample sizes.)

I have data from all 3,143 counties in the USA. For each county the population is given, and there is a list of variables which are counts or averages for that county, such as the number of gas stations, number of supermarkets, number of fast food restaurants, the stroke rate, the diabetes rate, the gender, and a person's race.

Scenario 1: Can I do a paired t-test on gender and the diabetes rate, or race and diabetes rate? (To show that the average varies between males and females, or Caucasions and colored people, across the whole data set.)

I found a couple of posts here that say a t-test isn't valid if the sample populations are significantly different in size, which is definitely the case here (LA county, vs. well, almost anywhere).

Scenario 2: Is there a way of testing for a significant relationship between fast food and diabetes, using some combination of the county population, number of fast food restaurants, fast food restaurant sales per capita, and the diabetes rate? I'm looking for a significance test that can be framed as a testable hypothesis, that uses a standard significance test (and not a data mining model).

• You do not want a t-test. You want some kind of correlation between fast food and diabetes.
– Dave
Apr 12 '20 at 21:19
• Okay. I'm looking for what statistical test to use.
– user252085
Apr 12 '20 at 21:23
• @Dave - I've added a second scenario to the question. The one I just added could use a paired t-test - if that's valid with varying population sizes.
– user252085
Apr 12 '20 at 21:35

## 2 Answers

So taking a step back, I think you might want to consider a different method from t-tests, altogether.

Your unit of analysis is county, and you have various statistics at the county level, including population, presumably what state it is in, health metrics (sounds as though you may have different stats for males/females even), and race breakdown. So this is a multivariate set of research questions and you almost certainly want to control for other properties of the county when making inferences about the target phenomena.

So with that in mind, I would toss out the idea of paired t-tests. Instead, I think mixed-effects models could be the best way go, and it should be configurable for both your scenarios above.

Let's take a quick toy data set as a starting point for Scenario 1:

    county state sex outcome
1 county 1    AL   M    0.65
2 county 1    AL   F    0.55
3 county 2    AL   M    0.45
4 county 2    AL   F    0.48
5 county 3    AL   M    0.71
6 county 3    AL   F    0.49
...


Using the lme4 package, and considering the properties of your outcome variable, you could set the model up as follows:

fit <- lmer(outcome ~ 1 + sex + (1 + sex|state), data=df)


Sometimes, referred to as a linear mixed effects model or a multilevel model, this approach would give you an estimate of the population-level difference in the outcome value for males and females in the U.S. One thing to call out is the (1 + sex|state) term in the model syntax. These are random effects and function as follows. When counties are in the same state, there is reason to believe that their average level (1 - the "intercept") of outcome and the association between sex and outcome are more similar on average than when counties are in different states. You may have heard the terms correlated errors or maybe "nested data." The flexibility of this modeling approach is that you can explicitly account for correlated errors using these random effect terms.

And lastly, there is a weights argument that you can use to adjust each county's contribution to parameter estimates based on population, which may be important given your analysis design.

• I have been looking AND looking AND looking for how to do weights, on several different problems for the past 3 weeks, and Google is giving me NOTHING. Please provide a link. Thank you. (The only thing I found was a mention of cell weights for surveys of different population sizes, apparently, in a commercial STATS (?) package, which I don't have.)
– user252085
Apr 13 '20 at 13:45
• rstudio-pubs-static.s3.amazonaws.com/… covers survey weights but the principle is largely the same for your use case I think. Apr 13 '20 at 14:15
• And for an overview and basic documentation of the package I mentioned checkout: cran.r-project.org/web/packages/lme4/vignettes/lmer.pdf Apr 13 '20 at 14:18
• Thanks. I will read through that when I have time later. Thanks.
– user252085
Apr 13 '20 at 17:43

Scenario 1: You can analyse the significance difference of diabetes rate between males and females or between Caucasians and Colored people. You would need to do Independent samples t-test and it does not require equal population in each of the categories. However, it should satisfy the equal variance assumption. If the equal variances are not fulfilled the procedure will be different. If you are using SPSS, you will get the values for both equal variances and unequaly variances will be available in the output. If you are using R, you need to first check whether the assumption is satisfied and then carryout the t-test with appropriate coding (specifying equal variances TRUE or FALSE).

t.test(diabetes ~ Gender, var.equal=TRUE, data=mydata)
t.test(diabetes ~ Gender, var.equal=FALSE, data=mydata)


Scenario 2: The relationship between fast food and diabetes can be modeled using linear regression, provided the assumptions (such as linearity, heteroscedasticity, no multi-collinearity) for the multiple linear regression are fulfilled. The linear variables can be identified by scatter plotting each of the predictor variables ti the diabetes rate.

P.S. Testing the equality of variances in R

library(car)
leveneTest(diabetes~Gender, center= mean, data=mydata)

• The per-capita number is already there. The problem is you have some counties with over a million people, and others with fewer than 50,000. So there are extremely different weights for the data values.
– user252085
Apr 12 '20 at 20:51
• @Victor updated my answer. Apr 12 '20 at 22:00
• Do you know of a reference that says the sample sizes can vary dramatically? (Scenario 1). For scenario 2, how can that be tested with null/alternative hypothesis?
– user252085
Apr 12 '20 at 22:15
• I understand that the population vary across counties, but not between males and females or between the races. Apr 12 '20 at 22:26
• Please see this post. stats.stackexchange.com/questions/223595/… They are saying a t-test is not valid when the sample sizes vary significantly. Answer 1: "provided the sample sizes are similar to each other." Answer 2: "This approach holds as long as the sample sizes do not get widely different."
– user252085
Apr 12 '20 at 22:33