# Combining data with differing dependent variables

Suppose we have two feature matrices, $$X_1$$ and $$X_2$$, with response variables $$Y_1$$ and $$Y_2.$$ Where $$X_1$$ and $$X_2$$ have the same feature columns, but distinct observations. Furthermore, $$Y_1$$ and $$Y_2$$ are continuous responses measured on different scales (i.e. the underlying distribution of $$Y_1$$ is different than the underlying distribution of $$Y_2$$). Suppose there exists a function $$f_Y$$ such that $$Y_1 \approx f_Y(Y_2)$$.

Can we combine the datasets $$(X_1, Y_1)$$ and $$(X_2, Y_2)$$ to increase the number of observations we have for regression?

For example, for the mappings $$\begin{matrix} X_1 & \underset{(1)}{\rightarrow} & Y_1 \\ & & \downarrow\tiny(Y) \\ X_2 & \underset{(2)}{\rightarrow} & Y_2 \\ \end{matrix}$$ One could use $$\underset{(1)}{\rightarrow}$$ and $$\underset{(2)}{\rightarrow}$$ to learn $$\downarrow\tiny(Y)$$ and then regress using transformed $$Y_2$$ values to get:

$$\left(\begin{matrix} X_1 \\ X_2 \\ \end{matrix}\right) \rightarrow \left(\begin{matrix} Y_1 \\ f_Y(Y_2) \\ \end{matrix}\right).$$

As an example of this problem in R, say we have the simulated data:

x1 = matrix(rnorm(500), nrow = 50)
x2 = matrix(rnorm(600), nrow = 60)
b1 = matrix(c(1,2,3, rep(0,7)), ncol = 1)
b2 = 2*(b1)^2
y1 = x1 %*% b1 + rnorm(50)
y2 = x2 %*% b2 + rnorm(60)


We can build simple linear regressions lm(y1~x1) and lm(y2~x2) to learn b1 and b2, but suppose we can learn a more powerful predictor if we combine the datasets.

How do we combine these data?

• Maybe write another equation of Y1 = f(Y2) and then use SUR? Aug 28 '19 at 12:07
• You haven't any information at all that could be used to estimate the mapping of dependent variables. To get some, you will need to make some assumptions; that is, create an explicit model of the responses. This is not a matter of combining data.
– whuber
Aug 28 '19 at 13:47
• @whuber do you think we could learn something from the crossed predictions $X2 \underset{(1)}{\rightarrow}$ and $X1 \underset{(2)}{\rightarrow}$? Aug 28 '19 at 14:33
• Yes, you might--but it is hard to see how you could improve either prediction using data from the other one without postulating some model about how they might be related.
– whuber
Aug 28 '19 at 14:52
• @whuber I see, so it's probably best to scale the response variables in a reasonable fashion, say percent rank, as opposed to trying to learn the "true" mapping. Aug 28 '19 at 15:40