I have a colleague that wants to fit a nonlinear model to the independent variables X (X is an n x k matrix) and the dependent variables Y (Y is an n x 1 matrix). The difficulty is that each row of Y could be from one of two different "types" of data, T1 and T2.

There are two different functional relationships between X and the two data types, but the functional relationship between X and the T1 data uses a subset of the parameters that the functional relationship between X and T2 does.

In the past, the nonlinear model between X1 and the T1 was fit and then the parameters fit in that first regression were fixed for the nonlinear regression between the X2 (note that this is a different set of observations) and T2 data.

My colleague is curious what happens if we fit the X1 , T1 data and the X2, T2 simultaneously (remember that T1 and T2 are different "types" of data with different scales and X1 and X2 are non-overlapping data sets). Is there a standard way to approach this problem? I think that I could standardized the residuals or something like that, however I don't really understand the theoretical justification for that and I feel that I could be making some massive error.


1 Answer 1


As I understand each $Y$ has type $T_{1}$ or $T_{2}$ and it is no row where two types are given simultaneously. Conseuqnently it is no theoretical grounds for assuming correlation (or some other kind of dependence) between errors in $X_{1},T_{1}$ and $X_{2},T_{2}$ models. Hence simultaneous estimation will give the same results.

Simultaneous estimations are reasonable when random components (usually errors) of estimated equations are dependend in some way (for example correlated) as it makes combinated log-likelihood function to be differerent from simple summ of marginal log-likelihood functions of each model.

For example suppose the model:

$\begin{cases}y_{1i}=\beta_{0}+\beta_{1}x_{1i}+\beta_{2}x_{2i}+\epsilon_{1i}\\ y_{2i}=\alpha_{0}+\alpha_{1}x_{1i}+\alpha_{2}x_{3i}+\epsilon_{2i}\end{cases}, \epsilon_{1*},\epsilon_{2*}\sim N(\begin{bmatrix}\mu_{1}\\\mu_{2}\end{bmatrix},\begin{bmatrix}\rho_{1}&\rho_{0}\\ \rho_{0}&\rho_{2}\end{bmatrix})$

Then if $\rho_{0}=0$ constistent estimates could be achieved without simultaneous estimation because log-likelihood function will be simple the summe of log-likelihood functions of univariate normal distributions. But if $\rho_{0}\ne0$ then log-likelihood function will be based on bivariate normal distributio, so it will have the different form and hence the different maximum that results in different outcome parameter vector. But in your case it is no such $i$ that both $y_{1i}$ or $y_{2i}$ (no matter how complicated your functional form, only how it meets the random component that is disturbance matters) observed simultaneously that is why I think it is no reason for simultaneous estimation. However more deep look at estimated equations unlikely, but maybe will give me some reasons to advice you simlultaneous estimation, but it is extremely unlikely as your case seems to have no dependence between random components of estimated equations.

  • $\begingroup$ However, in this case, the two models share three parameters, B<sup>(0)</sup>, B<sup>(1)</sup> and B<sup>(2)</sup>, which would suggest that there may be correlation between errors. (I think!) $\endgroup$
    – atchn01
    Dec 28, 2016 at 22:00

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.