I believe that independent variables $X_1,X_2$ affect the dependent variable $Y$ through a latent variable $Z$ such that $$ \begin{align} Y &= \beta_0 + \beta_1Z \\ Z &= \operatorname{Logit}^{-1}(\beta_2X_1 + \beta_3X_2) \\ \\ Y &= \beta_0 + \beta_1\operatorname{Logit}^{-1}(\beta_2X_1 + \beta_3X_2) \end{align} $$

Is it possible to estimate $\beta_2$ and $\beta_3$, given $Y$?

  • $\begingroup$ Expit="Inverse logit function" from the msm package? $\endgroup$ – chl May 21 '11 at 19:19
  • $\begingroup$ yes, I'll fix the question so that it's clear I'm talking about the inverse logit fuction. $\endgroup$ – fgregg May 21 '11 at 19:24
  • 2
    $\begingroup$ if $X_1$ and and $X_2$ are given the obvious choice is non-linear least squares. What are the conditions on model errors? Depending on them better estimating technique can be used. $\endgroup$ – mpiktas May 21 '11 at 19:44
  • 1
    $\begingroup$ note that with the current equations $Z$ is not binary variable. $\endgroup$ – mpiktas May 21 '11 at 19:44
  • $\begingroup$ excellent point, I am interpreting Z as an expectation that Z will take on one of two values, but Z is continuous from 0 to 1. $\endgroup$ – fgregg May 21 '11 at 19:53

One answer is "no." Another is, "of course."


To simplify notation, let $\lambda(x) = 1/(1 + \exp(-x))$, the inverse logit. Because $\lambda(x) = 1 - \lambda(-x)$,

$$\beta_0 + \beta_1 \lambda(x) = (\beta_0 + \beta_1) - \beta_1 \lambda(-x)).$$

Therefore it is impossible to distinguish the parameters $(\beta_0, \beta_1, \beta_2, \beta_3)$ from $(\beta_0+\beta_1, -\beta_1, -\beta_2, -\beta_3)$.

Of course

Let us stipulate that the first nonzero element of $(\beta_1, \beta_2, \beta_3)$ must be positive. That resolves the indeterminacy. We still need a model for the errors. If we suppose, for instance, that $Y - \left(\beta_0 + \beta_1 \lambda(\beta_2 X_2 + \beta_3 X_3)\right)$ has a Normal distribution and the various $Y$'s are independent, then we can use least squares to estimate the parameters. There is no exact solution to this nonlinear optimization problem, but it is straightforward to do numerically.

50 data points and their fit as a surface

This graphic shows 50 points generated with standard Normal values for $X_1$ and $X_2$, parameter $\beta = (1,2,1/2,-1)$, with iid Normal errors of standard deviation 1/2. The surface shows the fit, $\hat{\beta} = (2.68, -1.23, -0.89, 1.75) \sim (1.45, 1.23, 0.89, -1.75)$.

Least squares is the maximum likelihood with iid Normal errors. With another error distribution, use MLE directly. You can obtain asymptotic confidence intervals for the parameters in the standard ways.


Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.