Testing whether the conditional correlations/covariances differ between two groups

Question

I have two samples of variables $\{y_{1i},y_{2i},x_i,s_i\}$. Where $y_1$ and $y_2$ are binary variables, $x$ is a continuous variable and $s$ is a sample indicator, taking the value 0 in one sample and the value 1 in the other.

I can test whether means are different for $y_{1i}$ and $y_{2i}$ separately using a simple t-test or equivalently by estimating this linear equation using OLS$$y_i=\alpha+\beta x_i+\tau s_i + \varepsilon_i$$The regression has the advantage of providing a test of differences in the conditional means $E[y_i1|x_i,s_i=0]-E[y_i1|x_i,s_i=1]$

Now I also want to test whether the relationship between $y_1$ and $y_2$ is the same in both samples. For example I want to test whether:$$Corr[y_{1i},y_{2i}|x_i,s_i=0]=Corr[y_{1i},y_{2i}|x_i,s_i=1]$$ To me this seems like it should be a textbook statistical test, but I can't seem to find what the default procedure/test for this would be.

Answer 1

Since you have two binary response variables in your analysis, there are four possible outcomes taken over those two variables. Hence, it makes sense to use some kind of regression model with a discrete four-category response ---e.g., the multinomial logistic regression model. The model form for this model can be written as:

$$\begin{equation} \begin{aligned} \mathbb{P}(Y_{1i}=0,Y_{2i}=0 | x_i, s_i) &= \frac{1}{1 + e^{\beta_{01} x_i + \gamma_{01} s_i}+e^{\beta_{10} x_i + \gamma_{10} s_i}+e^{\beta_{11} x_i + \gamma_{11} s_i}}, \\[10pt] \mathbb{P}(Y_{1i}=0,Y_{2i}=1 | x_i, s_i) &= \frac{e^{\beta_{01} x_i + \gamma_{01} s_i}}{1 + e^{\beta_{01} x_i + \gamma_{01} s_i}+e^{\beta_{10} x_i + \gamma_{10} s_i}+e^{\beta_{11} x_i + \gamma_{11} s_i}}, \\[10pt] \mathbb{P}(Y_{1i}=1,Y_{2i}=0 | x_i, s_i) &= \frac{e^{\beta_{10} x_i + \gamma_{10} s_i}}{1 + e^{\beta_{01} x_i + \gamma_{01} s_i}+e^{\beta_{10} x_i + \gamma_{10} s_i}+e^{\beta_{11} x_i + \gamma_{11} s_i}}, \\[10pt] \mathbb{P}(Y_{1i}=1,Y_{2i}=1 | x_i, s_i) &= \frac{e^{\beta_{11} x_i + \gamma_{11} s_i}}{1 + e^{\beta_{01} x_i + \gamma_{01} s_i}+e^{\beta_{10} x_i + \gamma_{10} s_i}+e^{\beta_{11} x_i + \gamma_{11} s_i}}. \\[10pt] \end{aligned} \end{equation}$$

In this model form you have unknown coefficients $\beta_{01}, \beta_{10}, \beta_{11}$ that describe the effect of $x_i$ and you have coefficients $\gamma_{01}, \gamma_{10}, \gamma_{11}$ that describe the effect of $s_i$. This gives you a basic model you can fit to your data. If you are interested in testing a hypothesis relating to the correlation between the response variables then you will need to frame this hypothesis in terms of the underlying parameters of the model. To do this you will need to use some algebra to derive the relevant correlation expressions in terms of these parameters.

---

Finding the correlation formulae: With a bit of algebra, we can show that the true covariance between the two response variables is:

$$\begin{equation} \begin{aligned} \mathbb{Cov}(Y_{1i}, Y_{2i} | x_i,s_i) &= \mathbb{E}(Y_{1i} \cdot Y_{2i} | x_i,s_i) - \mathbb{E}(Y_{1i} | x_i,s_i) \cdot \mathbb{E}(Y_{1i} | x_i,s_i) \\[10pt] &= \mathbb{P}(Y_{1i} =1, Y_{2i}=1 | x_i,s_i) - \mathbb{P}(Y_{1i} =1 | x_i,s_i) \cdot \mathbb{P}(Y_{1i}=1 | x_i,s_i) \\[10pt] &= \frac{e^{\beta_{11} x_i + \gamma_{11} s_i} (1- e^{\beta_{11} x_i + \gamma_{11} s_i})}{(1 + e^{\beta_{01} x_i + \gamma_{01} s_i}+e^{\beta_{10} x_i + \gamma_{10} s_i}+e^{\beta_{11} x_i + \gamma_{11} s_i})^2}. \\[10pt] \end{aligned} \end{equation}$$

For the variances, with a bit more algebra we get:

$$\begin{equation} \begin{aligned} \mathbb{V}(Y_{1i} | x_i,s_i) &= \frac{(e^{\beta_{10} x_i + \gamma_{10} s_i} + e^{\beta_{11} x_i + \gamma_{11} s_i} )(1- e^{\beta_{10} x_i + \gamma_{10} s_i}- e^{\beta_{11} x_i + \gamma_{11} s_i})}{(1 + e^{\beta_{01} x_i + \gamma_{01} s_i}+e^{\beta_{10} x_i + \gamma_{10} s_i}+e^{\beta_{11} x_i + \gamma_{11} s_i})^2}, \\[10pt] \mathbb{V}(Y_{2i} | x_i,s_i) &= \frac{(e^{\beta_{01} x_i + \gamma_{01} s_i} + e^{\beta_{11} x_i + \gamma_{11} s_i} )(1- e^{\beta_{01} x_i + \gamma_{01} s_i}- e^{\beta_{11} x_i + \gamma_{11} s_i})}{(1 + e^{\beta_{01} x_i + \gamma_{01} s_i}+e^{\beta_{10} x_i + \gamma_{10} s_i}+e^{\beta_{11} x_i + \gamma_{11} s_i})^2}. \\[10pt] \end{aligned} \end{equation}$$

Hence, we obtain the correlation formula:

$$\begin{equation} \begin{aligned} \mathbb{Corr}(Y_{1i},Y_{2i} | x_i,s_i) &= \frac{(e^{\beta_{10} x_i + \gamma_{10} s_i} + e^{\beta_{11} x_i + \gamma_{11} s_i} )(1- e^{\beta_{10} x_i + \gamma_{10} s_i}- e^{\beta_{11} x_i + \gamma_{11} s_i})}{\sqrt{(e^{\beta_{10} x_i + \gamma_{10} s_i} + e^{\beta_{11} x_i + \gamma_{11} s_i} )(1- e^{\beta_{10} x_i + \gamma_{10} s_i}- e^{\beta_{11} x_i + \gamma_{11} s_i}) \\ \cdot (e^{\beta_{01} x_i + \gamma_{01} s_i} + e^{\beta_{11} x_i + \gamma_{11} s_i} )(1- e^{\beta_{01} x_i + \gamma_{01} s_i}- e^{\beta_{11} x_i + \gamma_{11} s_i})}}. \\[10pt] \end{aligned} \end{equation}$$

So the particular correlation formulae for $s_i=0$ and $s_i=1$ are:

$$\begin{equation} \begin{aligned} \mathbb{Corr}(Y_{1i},Y_{2i} | x_i,s_i=0) &= \frac{(e^{\beta_{10} x_i} + e^{\beta_{11} x_i})(1- e^{\beta_{10} x_i}- e^{\beta_{11} x_i})}{\sqrt{(e^{\beta_{10} x_i} + e^{\beta_{11} x_i} )(1- e^{\beta_{10} x_i}- e^{\beta_{11} x_i}) (e^{\beta_{01} x_i} + e^{\beta_{11} x_i})(1- e^{\beta_{01} x_i}- e^{\beta_{11} x_i})}}, \\[10pt] \mathbb{Corr}(Y_{1i},Y_{2i} | x_i,s_i=1) &= \frac{(e^{\beta_{10} x_i + \gamma_{10}} + e^{\beta_{11} x_i + \gamma_{11}} )(1- e^{\beta_{10} x_i + \gamma_{10}}- e^{\beta_{11} x_i + \gamma_{11}})}{\sqrt{(e^{\beta_{10} x_i + \gamma_{10}} + e^{\beta_{11} x_i + \gamma_{11}} )(1- e^{\beta_{10} x_i + \gamma_{10}}- e^{\beta_{11} x_i + \gamma_{11}}) \\ \cdot (e^{\beta_{01} x_i + \gamma_{01}} + e^{\beta_{11} x_i + \gamma_{11}} )(1- e^{\beta_{01} x_i + \gamma_{01}}- e^{\beta_{11} x_i + \gamma_{11}})}}. \\[10pt] \end{aligned} \end{equation}$$

The requirement that $\mathbb{Corr}(Y_{1i},Y_{2i} | x_i,s_i=0) = \mathbb{Corr}(Y_{1i},Y_{2i} | x_i,s_i=1)$ means that you equate these two formulae and find the conditions on the parameters that lead to this equation. (Algebraically, it is easier to equate the squared-correlations, which simplifies the expression.) Since you want this equation to hold for all $x_i$ this will impose quite strict requirements on the parameters.

A sufficient condition for equivalence is to have $\gamma_{01} = \gamma_{10} = \gamma_{11} = 0$ so that the variable $s_i$ has no effect at all on the responses. (In other words, it is not really a distinct test in that case.) It is not obvious whether there is any weaker condition that would yield the required correlation equivalence.

Answer 2

EDIT: This is my attempt assuming the outcomes are joinly normal, instead of binary

First, I phrase my assumptions about the joint distribution as: $$\left(\begin{matrix}y_1\\y_2\end{matrix}\right) \sim N\left(\mu,\Sigma\right),$$ with $$\mu=\left(\begin{matrix}\mu_1+\gamma_1s+\delta_1x\\\mu_2+\gamma_2s+\delta_2x\end{matrix}\right)$$ and $$\Sigma=\left(\begin{matrix}\sigma^2_1&,& \alpha+\beta_1 x + \beta_2 s\\\cdot&,& \sigma^2_2\end{matrix}\right)$$

Testing whether the covariance differs by sample is equivalent to testing that $\beta_2=0$. Do do that, I estimate $$Cov[y_{1i},y_{2i}|x_{i},s_i] =\\ E[(y_{1i}-E[y_{1i}|x_{i},s_i])\cdot(y_{2i}-E[y_{2i}|x_{i},s_i])|x_{i},s_i]=\alpha+\beta_1x_i+\beta_2s_i \quad\quad\quad\quad\quad\quad$$ by:

1. fitting a simple linear (or logit) model to estimate $E[y_{1i}|x_{i},s_i]$ and $E[y_{2i}|x_{i},s_i]$.
2. obtain the fitted values from this and compute the expression: $$\widehat{cov_i}=(y_{1i}-\widehat{E[y_{1i}|x_{i},s_i]})\cdot(y_{2i}-\widehat{E[y_{2i}|x_{i},s_i]})$$
3. estimate the model: $$\widehat{cov}_i=\alpha+\beta_1x_i+\beta_2s_i + \varepsilon_i$$

This seems to work well. If correlations are of interest instead of covariances, step 2. can be augmented dividing the $\widehat{cov_i}$ by similarly obtained estimators for the two standardeviations.