# Univariate approach to a Bivariate logistic regression

Consider a situation where two independent agents (out of a set of many agents) look at the same problem and attempt to solve it with a yes/no response, obtaining $$(Y_{i1},Y_{i2})$$ for $$i \in \{1,\ldots,N\}$$. Each agent is highly trained, and they often come to the same conclusion. If we had a set of covariates of the task $$X_i$$ which influence each decision that we wish to gather information about, this could be approached as a multivariate problem simultaneously modelling $$(Y_{i1},Y_{i2})$$ using a logit or probit link function. The 'agreement' between the two agents would be inferred from the covariance matrix between the two latent response variables. In the probit example, this would resemble:

$$(Y_1^*,Y_2^*) \sim MVN((\mu_1,\mu_2),\Sigma)$$

where $$Y_i = \{1 \text{ if } Y_1^* > 0, \text{ 0 otherwise}\}$$, $$\Sigma = \left(\begin{matrix} \sigma_1 & \sigma_{12}\\ \sigma_{12} & \sigma_2 \end{matrix}\right)$$ and $$\mu_i = \beta X_i$$.

If a multivariate approach were for some reason infeasible (in my case due to software incompatibility with other modelling choices), is it possible to approach the problem by modelling each decision independently by combining the response vectors into a single $$Y_j$$ for $$j \in 1,\ldots,2N$$ response and using factor variables to distinguish between the two? For example the probit regression would then resemble:

$$\mu_j = \beta_0 + \beta_1 \text{Other decision}_j + \beta_3 \text{First or Second}_j + \beta X_j$$ for $$j \in \{1,\ldots,2N\}$$ using a regular normal distribution.

In this approach the $$\text{First or second}_j$$ variable effectively models the different intercepts of each agents' acceptance rate and the $$\text{Other decision}_j$$ models the strength of agreement between the two agents. Is this, or any, univariate approach equivalent to solution to the bivariate problem? I would think that the only difference between them is the choice to combine or partition the elements of variance but I cannot find any references to this being done or considered before.

Let's see if this leads anywhere. From an econometrics point of view, I see two latent regressions,

\begin{align} \begin{cases} y^*_1 = \beta x_1 + u_1\\ y^*_2 = \beta x_2 + u_2 \end{cases} \end{align} with conditional marginal distributions \begin{align} \begin{cases} u_1 \mid X \sim N(0,1)\\ u_2 \mid X \sim N(0,1)\end{cases} \end{align}

The variances have been standardized to unity since the $$\beta$$ coefficients are identifiable only up to scale (due to the linearity of the link function). The additional information we have is that the two $$u$$-errors are correlated as a bivariate Normal, and here their covariance is equal to their correlation coefficient $$\rho$$, since variances are unitary: \begin{align} (u_1,u_2) \mid \{x_1, x_2\} & \sim MVN((0,0),\Sigma_u)\\ \\ &\Sigma_u = \left(\begin{matrix} 1 & \rho\\ \rho & 1 \end{matrix}\right) \end{align}

The problem we face is that the additional information is this correlation between the two errors, and we do not have data on them. Ignorance of this correlation does not destroy the consistent estimation of the $$\beta$$ coefficients from the marginal models, but presumably, in finite samples it would be better to use all the data together, alongside the correlation. To arrive there, set as usual

\begin{align} \begin{cases} y_1 \equiv I\{y^*_1>0\} \\ y_2 \equiv I\{y^*_2>0\} \end{cases} \end{align}

i.e the indicator functions on which we have data alongside $$X$$.

Then we can consider the following conditional probability model (suppressing the conditioning on $$X$$, considering it understood),

\begin{align} \Pr(y_1=1 \mid y_2 = 1) & = \Pr(\beta x_1 + u_1 > 0 \mid \beta x_2 + u_2 > 0)\\ \\ &= \frac{\Pr(\beta x_1 + u_1 > 0\, , \, \beta x_2 + u_2 > 0)}{\Pr(\beta x_2 + u_2 > 0)}\\ \\ &=\frac{\Pr( u_1 > -\beta x_1 \, , \, u_2 > -\beta x_2)}{1-\Phi\left(-\beta x_2\right)}\\ \\ &=\frac{\Phi_2(\beta x_1 \, , \, \beta x_2\,;\,\rho)}{\Phi\left(\beta x_2\right)} \end{align}

We have used the radial symmetry of the bivariate Normal. $$\Phi_2$$ is the bivariate Normal integral, and $$\Phi$$ the Normal distribution function. Both are implemented as readymade special functions in most computational software with a statistical flavor.

We can formulate the other three events combining $$y_1$$ and $$y_2$$ in an analogous manner, and then we can form the full likelihood for $$\Pr(y_1 \mid y_2)$$ and proceed as usual... and then we have to consider $$\Pr(y_2 \mid y_1)$$ (they are not symmetric due to the different denominator, since the OP indicates that the $$X$$-data (realizations) related to $$y_1$$ are not the same as the $$X$$-data related to $$y_2$$).

This is a way to take into account the correlation between the two underlying error terms, in a "univariate" setting, using all the data available together and obtain a common estimate for the $$\beta$$ vector (and of $$\rho$$)... although the use of $$\Phi_2$$ in estimation algorithms in practice may encounter difficulties.

It would be interesting to compare the obtained as above $$\beta$$ estimates to the estimates than one would get from the two marginal models. In principle, all will be consistent estimates, but in finite samples, who knows...

This should be possible with bivariate probit regression. The observed bivariate binary response can be represented via thresholding a bivariate normal latent variable. A bivariate normal with dependent components can be represnted as $$(Y_{i1}^*, Y_{i2}^*) = (Z_{i1}+Z_{i0}, Z_{i2}+Z_{i0})$$ with $$Z_{i1}, Z_{i2}$$ independent. The common component $$Z_{i0}$$ which creates the dependence will then enter the model as a latent variable, that is, a random effect.

So what you get is a nonlinear mixed effects model. In R, for instance, it could be fitted using lme4 package, using a binomial family with probit link function.

I will add details later (traveling).

• Thank you for your answer Kjetil. My question is whether there is an equivalent univariate approach to the problem that captures the same information as the bivariate approach - I've edited the question to reflect this. Jan 19 at 14:03
• Well, what I have given is a univariate approach, but it requires mixed models! Jan 19 at 14:44