Converting Categorical Data to Numerical by Sampling

Question

Suppose I had sampled $1000$ individuals from a population in order to learn about two different questions, both of which had categorical, binary answers. (For the sake of this hypothetical, let's say those two variables are the responses to the questions "Do you have a car?" and "Do you change your outdoor dining behaviors in the winter?") The results of any individual could therefore be mapped into the Cartesian plane as $(0,0), (1,0), (0,1), (1,1)$ ("no" = $0$, "yes" = $1$), and suppose that I want to create a regression to see if there is a correlation between my $x$ and $y$ values. I know that we could create $2\times 2$ contingency table and run a logistic regression on that data based on the total responses. However, I was curious if we could take arbitrarily many samples from this sample with replacement (let's say $1000$ samples of size $10$), and then map the frequencies of the two variables on a plane and regress that data. In other words, we would have data ranging from $(0,0)$ to $(10,10)$, with all values in between. My question is whether or not this is a legitimate way to draw inferences from our data set, and whether or not this "conversion" from categorical to numerical data actually provides a relevant picture as to the relationship of our two original variables. Any insights would be sincerely appreciated. Thanks!

Answer 1

This strikes me as a good way to add unnecessary noise to an analysis. We can study the noise with a simulation in R:

mod_cor <- reg_cor <- rep(NA, 400)

for (i in 1 : 400) {
  dat <- data.frame(x1 = rbinom(1000, 1, 0.8),
                    x2 = rbinom(1000, 1, 0.3))

  reg_cor[i] <- with(dat, cor(x1, x2))
  dat_samp <- dat[FALSE, ]
  for (j in 1 : 1000) {
    dat_samp[j, ] <- apply(dat[sample(1 : 1000, 10, replace = TRUE), ], 2, sum)
  }
  mod_cor[i] <- with(dat_samp, cor(x1, x2))
}

quantile(reg_cor, c(0.025, 0.975))
quantile(mod_cor, c(0.025, 0.975))

I made an uncorrelated data set of size $1000$, and compared your proposed resampled data with the original data on the basis of a correlation coefficient. The added noise can be studied by looking at the distributions of the correlation coefficients (or other test statistics, but I just chose this one because you mentioned you want to do regression). For example, 95% intervals are

# sim interval for correlation with original data
       2.5%       97.5% 
-0.06432026  0.05491876 

# sim interval for correlation with resampled data
       2.5%       97.5% 
-0.08936199  0.08227468 

Resampling with sets of size 10 increases the length of the uncertainty interval by about 45%.

Another angle is the resampling takes your $1000$ possibly independent observations and makes them dependent in some weird way.