Can a significance test be done on two variables which are 3,140 county averages? (Significantly different sample sizes.)

Question

I have data from all 3,143 counties in the USA. For each county the population is given, and there is a list of variables which are counts or averages for that county, such as the number of gas stations, number of supermarkets, number of fast food restaurants, the stroke rate, the diabetes rate, the gender, and a person's race.

Scenario 1: Can I do a paired t-test on gender and the diabetes rate, or race and diabetes rate? (To show that the average varies between males and females, or Caucasions and colored people, across the whole data set.)

I found a couple of posts here that say a t-test isn't valid if the sample populations are significantly different in size, which is definitely the case here (LA county, vs. well, almost anywhere).

Scenario 2: Is there a way of testing for a significant relationship between fast food and diabetes, using some combination of the county population, number of fast food restaurants, fast food restaurant sales per capita, and the diabetes rate? I'm looking for a significance test that can be framed as a testable hypothesis, that uses a standard significance test (and not a data mining model).

Answer 1

So taking a step back, I think you might want to consider a different method from t-tests, altogether.

Your unit of analysis is county, and you have various statistics at the county level, including population, presumably what state it is in, health metrics (sounds as though you may have different stats for males/females even), and race breakdown. So this is a multivariate set of research questions and you almost certainly want to control for other properties of the county when making inferences about the target phenomena.

So with that in mind, I would toss out the idea of paired t-tests. Instead, I think mixed-effects models could be the best way go, and it should be configurable for both your scenarios above.

Let's take a quick toy data set as a starting point for Scenario 1:

    county state sex outcome
1 county 1    AL   M    0.65
2 county 1    AL   F    0.55
3 county 2    AL   M    0.45
4 county 2    AL   F    0.48
5 county 3    AL   M    0.71
6 county 3    AL   F    0.49
...

Using the lme4 package, and considering the properties of your outcome variable, you could set the model up as follows:

fit <- lmer(outcome ~ 1 + sex + (1 + sex|state), data=df) 

Sometimes, referred to as a linear mixed effects model or a multilevel model, this approach would give you an estimate of the population-level difference in the outcome value for males and females in the U.S. One thing to call out is the (1 + sex|state) term in the model syntax. These are random effects and function as follows. When counties are in the same state, there is reason to believe that their average level (1 - the "intercept") of outcome and the association between sex and outcome are more similar on average than when counties are in different states. You may have heard the terms correlated errors or maybe "nested data." The flexibility of this modeling approach is that you can explicitly account for correlated errors using these random effect terms.

And lastly, there is a weights argument that you can use to adjust each county's contribution to parameter estimates based on population, which may be important given your analysis design.

Answer 2

Scenario 1: You can analyse the significance difference of diabetes rate between males and females or between Caucasians and Colored people. You would need to do Independent samples t-test and it does not require equal population in each of the categories. However, it should satisfy the equal variance assumption. If the equal variances are not fulfilled the procedure will be different. If you are using SPSS, you will get the values for both equal variances and unequaly variances will be available in the output. If you are using R, you need to first check whether the assumption is satisfied and then carryout the t-test with appropriate coding (specifying equal variances TRUE or FALSE).

t.test(diabetes ~ Gender, var.equal=TRUE, data=mydata)
t.test(diabetes ~ Gender, var.equal=FALSE, data=mydata)

Scenario 2: The relationship between fast food and diabetes can be modeled using linear regression, provided the assumptions (such as linearity, heteroscedasticity, no multi-collinearity) for the multiple linear regression are fulfilled. The linear variables can be identified by scatter plotting each of the predictor variables ti the diabetes rate.

P.S. Testing the equality of variances in R

library(car)
leveneTest(diabetes~Gender, center= mean, data=mydata)