Let
$$\begin{array}{} Y_i & \sim& Bernoulli(0.5) \\ X_i|Y_i &\sim& N(\mu_{Y_i},\sigma^2) \end{array}$$
In this case we can consider independent pairs of observations $X_i,Y_i$ following the model of a logistic regression where the $Y_i$ has a conditional Bernoulli distribution with
$$P(Y_i=1|X_i=x) = \frac{1}{1+e^{-(a+bx)}}$$
where $a = \frac{\mu_1^2-\mu_0^2}{2\sigma^2}$ and $b=\frac{\mu_0-\mu_1}{\sigma^2}$
So, we can estimate these parameters $a$ and $b$ in two different ways
- Perform logistic regression.
- Estimate $\mu_i$ and $\sigma$ directly with the assumed normal distribution, then compute to the parameters $a$ and $b$.
The two methods will not give the same result. Which method is most efficient (has the expected squared error)?
To clarify my question a bit further. It is also about understanding.
Here is a simulation that shows that the estimate of mean and variance results in smaller error. And it could answer the question.
- But is it general?
- And why wouldn't logistic regression perform the same?
- What sort of information does the knowledge about the distribution of $X_i$ add and could this be incorporated into the logistic regression (e.g. by adding weights) to make it perform better?
set.seed(1)
sim = function(n, mu) {
Y = rbinom(n,1,0.5)
X = rnorm(n,Y*mu,1)
mod = glm(Y ~ X, family = binomial)
glm_est = mod$coefficients
m = c(mean(X[Y==0]), mean(X[Y==1]))
V = (sum((X-m[Y+1])^2)/(n-2))
mm_est = c(-diff(m^2)/2/V,
diff(m)/V)
return(c(glm_est, mm_est))
}
mu = 1
z = replicate(10^3, sim(200, mu))
true_a = -mu^2/2
true_b = mu
### glm_errors
mean((z[1,]-true_a)^2) # 0.03592216
mean((z[2,]-true_b)^2) # 0.03245813
### gaussian_mm_errors
mean((z[3,]-true_a)^2) # 0.01323784
mean((z[4,]-true_b)^2) # 0.03075781
b
. Addmethod = brglm2::brglm_fit
to implement this. This makes the MSE forb
smaller than when using the direct calculation. Fora
I still get better results using the direct calculation, even when using Firth with intercept correction (logistf::flic()
). $\endgroup$