I have a theoretical model and N runs of an experiment, where in each outcome of the experiment I observe the values of three variables: Y1, Y2 and Y3=1-Y1-Y2.

According to my theoretical model, the values of Y1, Y2 and Y3 are as follows: Y1=f1(alpha); Y2=f2(alpha) and Y3=1-Y1-Y2, where f1, f2 are non-linear functions of the parameter alpha.

My goal is to estimate alpha. If I had only Y1, then I would have run a single non-linear regression and get the estimate. What should I do in the case with Y1, Y2 and Y3?


1 Answer 1


If Y3 is always exactly equal to one minus the sum of Y1 and Y2, as I think you are saying, then provide it no further consideration.

You can treat this as a multivariate regression problem (i.e., 2 or more possibly dependent outputs per "input" data point), for which there are many links on this forum.

If the errors are uncorrelated between Y1 and Y2 for any input data point, then you can solve this as a nonlinear least squares problem, just lumping together all the Y1 and Y2, i.e. $2n$ residuals if you have $n$ sets of Y1, Y2 (and Y3) observations. However, if the errors (distribution) are bigger for one of Y1 and Y2 than the other, then you can weight the Y1 vs. the Y2 residuals accordingly (which is a special case of what id described in the following paragraph), i.e., weighted least squares.

More generally, if the errors in Y1 and Y2 are not uncorrelated for any given data point, you can incorporate this correlation into a weighting matrix, $W$, equal (or proportional) to the inverse of the error covariance matrix $C$ and use generalized least squares.. If you stack all $2n$ residuals into a vector $r$, you would minimize $\frac{1}{2}r^TWr$. If you let $R$ be the upper triangular Cholesky factor of $C$, i.e., such that $R'R = C$, then you could form the adjusted residuals $r_{adj} = R^{-1}r$, for which the residuals would have covariance matrix equal to the identity, resulting in the problem, minimize $\frac{1}{2}r_{adj}^Tr_{adj}$. By taking this latter approach, you can apply standard "off the shelf" nonlinear least squares software to the quantity $2n$ of adjusted residuals.

So the key is paying attention to the relation of the errors across Y1 and Y2, and more generally across all the data points, and proceeding accordingly.

  • $\begingroup$ Thank you very much for the answer! Just to make it even more clear for a person like me, who is far from all this stuff: to check if the error terms are uncorrelated between Y1 and Y2, I need to run two separate regressions (one for Y1 and the other one for Y2) and then compare the obtained distributions of error terms with each other, right? $\endgroup$
    – 11110000
    Commented Jun 10, 2018 at 19:05
  • $\begingroup$ You should really try to base correlation structure on a priori information (such as knowledge of how the observations are created and measured), if you can. If you perform separate regressions with Y1 and Y2, you will see whether errors are of comparable magnitude, and if not, then weight the residuals for Y1 vs. Y2 in the pooled regression. Results will not be biased if covariance of error is not accounted for, but variance in estimated alpha may be larger than it could be. $\endgroup$ Commented Jun 10, 2018 at 19:24
  • $\begingroup$ Great! Thanks a lot again, and sorry for the disturbance from my side! $\endgroup$
    – 11110000
    Commented Jun 10, 2018 at 19:29

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.