I have a dataset $D = (x_i, y_i)_{i=1}^n$ where $x_i\in \Bbb R^d$ and $y_i\in\{0, 1\}$. Suppose that $y\sim\mathrm{Bernoulli}(p(x))$ for some probability function $p:\Bbb R^d \to [0,1]$ and I would like to learn this function the best way I can.
Would that be correct to say that logistic regression is actually focused on approximating $p$ with some predictor $q$ which minimizes the cross-entropy loss $\mathrm{CE}(p,q)$?
What are the other useful methods to approximate $p$ based off the data $D$? Of course I can replace a linear model in logistic regression with some non-linear counterpart $q_\theta:\Bbb R^d \to [0,1]$ and look for $\theta$ that minimizes $\mathrm CE(p,q_\theta)$, searching for example over NNs or random forests or whatever other non-linear point estimators come to mind. Perhaps, some Bayesian frameworks would come handy here?
If I need to learn the entropy of $p(x)$ at a given $x$ of course I can just approximate it with the entropy of $q(x)$: if $p\approx q$ then we should expect $H(p) \approx H(q)$. I wonder however whether there is a direct way to learn $H(p)$ as a function of $x$, perhaps with me wanting to minimize $\mathrm{MSE}(H(p), f_\theta)$ where $f_\theta$ is some parametrized positive-valued function.