Consider the logistic regression where $Y_i \in {0,1}$ are dependent variable observations and $X_i \in \mathbb{R}$ are the independent variables.

However we do not observe the $X_i$ themselves. Instead we observe some vector of parameters $\boldsymbol{\mu}_i$ and we know the distribution $F$ s.t. $F(X_i|\boldsymbol{\mu}_i)$.

How can we perform a logistic regression where the $X_i$ are not themselves observed? For $F$ a general distribution, one way may be to sample for each $i$, many values from $F(X_i|\boldsymbol{\mu}_i)$ and put them all into the regression model as observations.

Can we do things more efficiently if we know for example that $F$ is a normal distribution?


I think you can also go for a maximum likelihood approach considering the $x_i$ are latent variables over which you marginalize a likelihood.

Let's say the likelihood of your usual logistic regression, if you oberved the $x$ values, is $\mathcal{L}(\beta, x, y)$ where $\beta$ is the vector of parameters (typically, $\mathcal{L}(\beta, x, y) = (\frac{1}{1 + e^{-\beta x}})^y (\frac{1}{1 + e^{\beta x}})^{1 - y}$).

Then the likelihood only observing $\mu$ and $y$ is $$\mathcal{L}(\beta, y, \mu) = \mathbb{E}_{X \sim F_{\mu}}[\mathcal{L}(\beta, y, X)]$$ And the total likelihood is just the product of the likelihoods over all observed $(y_i, \mu_i)$.

Unfortunately, these expectations may be untractable (maybe for a simple normal distribution it is not, but it is not obvious to me...), so you can estimate them by Monte Carlo. For instance, sample $x_i \sim F_{\mu_i}$ and take empirical mean of $\mathcal{L}(\beta, y_i, x_i)$. I don't think that this is equivalent to simulating data according to $F_{\mu_i}$ and put them into the model, but it would be nice to see the links...

Another way would be to go with an E-M algorithm (where $x_i$ are the latent variables) to maximize this likelihood, this would vertainly be more computationally efficient.

I hope this helps a little bit...


Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.