If I have data $\{(x_i, y_i)\}_{i=1}^n$ where the dependent variable is binary $(y_i = 0,1)$ I can model it using a logistic function:
$$f(x; \alpha, \beta) = \frac{1}{1 + e^{-(\alpha + \beta x)}}$$
One way to estimate the parameters is by using the likelihood function,
$$L(\alpha,\beta) = \prod_{i=1}^n{f(x_i;\alpha,\beta)^{y_i}(1 - f(x_i;\alpha,\beta))^{(1-y_i)}}$$
and maximizing it using gradient ascent over $\alpha$ and $\beta$, since no closed form for the critical point exists.
I've been reading a bit on information geometry and have seen that for probability distributions it is more effective (in terms of the number of steps to converge) to use the natural gradient given by the KL-divergence and Fisher Information matrix, since it better captures the notion of distance between two distributions. Is there a similar notion of divergence between two logistic functions that could be leveraged to define a more efficient gradient? Is the Euclidean distance in $\mathbb{R}^2$ between the parameters $\alpha$ and $\beta$ the best way to traverse the parameter space?
