3
$\begingroup$

I have a data set in which female is coded as 0 and male as 1 and then I have different testing scores; writing, reading, math. I want to determine whether the correlation between math and writing scores significantly differs between the male and female populations. How would I code this in R? Note: My sample sizes for females is greater than males.

$\endgroup$
2
  • $\begingroup$ Essentially you could frame it as a hypothesis, with the null that the there is no interaction between gender and math in predicting writing scores. Then fit the regression E[Writing | Gender, Math] = a + b Writing + c Math + d Writing Math. If the coefficient d is statistically significantly different from 0, you can reject the null. $\endgroup$
    – AdamO
    May 5, 2022 at 19:59
  • 1
    $\begingroup$ The coefficient d can be significantly different from 0 while the correlations are the same. This can happen when the genders do not have the same variance in scores of math or writing. $\endgroup$ May 5, 2022 at 20:52

2 Answers 2

4
$\begingroup$

To show how you could approach this problem with a Monte Carlo simulation (as suggested in Sextus Empiricus's answer)...

You could either do it with a permutation test (AKA randomisation test) or with bootstrapping. I have chosen the latter.

First, I created some fake data that should look somewhat like yours:

library(tidyverse)

set.seed(123)

SAMPLE_SIZE <- 1000
MALE_PROP <- 0.4
n_males <- SAMPLE_SIZE * MALE_PROP
n_females <- SAMPLE_SIZE * (1 - MALE_PROP)

males <- data.frame(
  gender = rep(1, n_males),
  writing = rnorm(n = n_males, mean = 60, sd = 10)
) %>% 
  mutate(math = writing * 0.80 + rnorm(n = n_males, mean = 5, sd = 5))

females <- data.frame(
  gender = rep(0, n_females),
  writing = rnorm(n = n_females, mean = 60, sd = 10)
) %>% 
  mutate(math = writing * 0.65 + rnorm(n = n_females, mean = 5, sd = 5))

df <- males %>% 
  bind_rows(females)

We have 1,000 individuals, 40% of whom are males. Here is a sample of 5 rows:

gender writing math
0 56.47547 46.80284
1 62.11980 50.17477
0 65.14230 51.73375
0 66.81830 55.42685
1 64.26464 55.45913

And this is what the correlations between writing and math scores look like for the two genders:

df %>% 
  ggplot(aes(x = writing, y = math, colour = factor(gender))) +
  geom_point() +
  geom_smooth(method = 'lm', colour = 'black') +
  facet_wrap(~ gender) +
  labs(colour = 'gender')

Raw correlations

Note that you can calculate the actual sample correlations by doing:

cor(males$writing, males$math)
# [1] 0.8460947
cor(females$writing, females$math)
# [1] 0.8002061

so we see that males have a slightly higher correlation. But is this difference statistically significant?


Now, here is where the magic happens. We draw 10,000 bootstrap resamples (with replacement) for each gender, calculate the respective correlations, and store the differences between male and female correlations:

male_rows <- df %>% 
    filter(gender == 1) %>% 
    select(writing, math)

female_rows <- df %>% 
    filter(gender == 0) %>% 
    select(writing, math)

N_BOOT <- 10000

corr_deltas <- numeric(N_BOOT)

for (i in 1:N_BOOT) {
  male_boot <- male_rows %>% 
    slice_sample(prop = 1, replace = TRUE)
  
  female_boot <- female_rows %>% 
    slice_sample(prop = 1, replace = TRUE)
  
  male_corr <- cor(male_boot$writing, male_boot$math)
  female_corr <- cor(female_boot$writing, female_boot$math)
  
  corr_delta <- male_corr - female_corr
  
  corr_deltas[i] <- corr_delta
}

Now we have all we need. We can plot a histogram of the correlation differences together with their 95% confidence interval:

lower_bound_95ci <- quantile(corr_deltas, 0.025)
upper_bound_95ci <- quantile(corr_deltas, 0.975)

ggplot() +
  geom_histogram(aes(x = corr_deltas)) +
  geom_vline(aes(xintercept = lower_bound_95ci), linetype = 'dashed') +
  geom_vline(aes(xintercept = upper_bound_95ci), linetype = 'dashed') +
  labs(title = paste(
    'The 95% CI for the difference in correlations is (',
    round(lower_bound_95ci, 3), ',', round(upper_bound_95ci, 3), ')'),
    subtitle = 'Therefore, males have a higher correlation than women.',
    x = 'correlation delta')

Bootstrap distribution

Since the 95% CI does not include 0 in this case, we would conclude that the correlation between writing and math scores was significantly higher for males than for females.

$\endgroup$
2
$\begingroup$

If your population is approximately normal distributed then you can approximate the distribution of a correlation coefficient by a normal distribution after applying a Fisher transformation.

$$z = 0.5 \log \frac{1+r}{1-r} \sim N\left(\mu,\sigma^2\right)$$

where $\mu = 0.5 \log \frac{1+\rho}{1-\rho}$ and $\sigma = \frac{1}{\sqrt{N-3}}$ where $\rho$ is the true correlation of the population and $N$ is the sample size.

Then you can take the difference of the two $z$ scores (based on your two correlations) and given the hypothesis that you have the same correlation the difference should be normal distributed with zero mean and standard deviation $\sqrt{\frac{1}{N_1-3}+\frac{1}{N_2-3}}$.


If your population is not approximately normal distributed, then you could approximate the distribution with a Monte Carlo method like resampling and observing the variation that occurs among the different samples. This works better if you have a large sample size.

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.