Guidelines for doing Maximum likelihood for nonlinear classification? I am using logistic regression for a problem and a colleague suggested trying a MLE solution, with the assumption that some of my features $X_i$ may not have a linear relationship with the target variable $Y_i$. 
So his idea is that I take my features (assume they are continuous) and "bin" them into $N_i$ buckets. Then calculate the proportions of members of the positive and negative classes that fall into each bucket, which gives a sort of non-parametric likelihood distribution (not sure if that's the right term to be honest). Once these nonparametric distributions are calculated, I could then calculate the likelihood ratio for a new (test) instance.
Having said all that, I was hoping I could find some references that illustrate this approach more formally than the mental model I currently have. Also, simply, does it make sense to do this kind of model? I don't want to spin my wheels. Are there any pitfalls I should look out for?
Also, I guess one question is whether I should just be using Random Forest? Wouldn't that be able to capture non-linearities just as well as the above approach?
 A: Without knowing the exact procedure in question it's a bit hard to critique, but at this point I'm pretty confident in saying that you have a lot of better tools at your disposal than what is being suggested and you'd be better off not doing it.
It sounds like you want to estimate the conditional distribution $P(Y=1 | X = x)$. A logistic regression does this by making additivity and linearity assumptions. You're considering an alternative approach where you change this from being continuous to instead being discrete, and then just use sample averages to estimate this probability within each cell of the $N_1 \times \dots \times N_p$ grid that now supports your distribution.
If that's really the proposed estimator, there are a number of fatal flaws. First, how could you produce an estimate for a new point that's not exactly like other ones you saw? Second, the vast majority of cells will be completely empty, and the ones that aren't will probably only have a couple of data points. Each individual cell's estimates will be terrible.
The only way to save this is to make some structural assumptions like independence, so you can attempt to recreate this joint distribution using lower-variance marginal information (I believe one such method is iterative proportional fitting where you use lower dimensional marginals to approximate a joint distribution table). This could also be done by fitting a logistic regression on the binned features, where now we can estimate each cell's probability by pooling information and we've made simplifying assumptions about the distribution.
But now we're just going back to where we started, and I think we're worse off than if we didn't bin at all and just fit a more flexible model. Binning is generally a bad idea: don't do it unless you have to (like if there are industry requirements or something). 
From what you've described I'm not sure how this is different from any other black box machine learning prediction problem, so unless there's something I'm missing there's really no need to cook up something new and questionable here. Continuous features are fantastic for fitting flexible models. If you mainly care about prediction why not use that random forest? That's perfect for this sort of thing. Or use a GAM, Gaussian process, or any other appropriate model that you like. 
It's definitely possible that I've missed something here, and what with no free lunch it's not guaranteed that this model would perform poorly in your setting, but I wouldn't bet on it doing well.
A: 
Hoping I could find some references that illustrate this approach more formally than the mental model I currently have

I think there are a lot of research papers addressing the binning of continuous variables (here's one: http://www.m-hikari.com/ams/ams-2014/ams-65-68-2014/zengAMS65-68-2014.pdf), however this approach is relatively rare (at least in my opinion) in machine learning approaches due to much more manual work needs to be done to create great binning.
Binning is usually (very often) used in actuarial credit, risk, premium models due to the nature that the industries are highly regulated and the model output need to make sense, especially since the most popular model in actuarial science is logistic regression, the linearity relationship between each variable in the model and the reponse need to be assessed with much much more care. And the approach is binning to create reasonable nominal variables to find the best linear relationship between the variables.
It also serves as a way to reduce the effect of extreme values as well.
However, the downside is that it will really take your time to assess each variable to create bins. And this needs to be done back and forth for all variables in your model.

Whether I should just be using Random Forest?

Short answer, you can. If you have no restrictions on which model to use, random forest will typically give you better result as it will handle the non-linearity of your data. However, this is going to reduce the interpretability of your model output compare to logistic regression; if this doesn't matter to you, go ahead.
A: So I gave this a go and found that I could use a kernel density estimator to calculate smooth pdf for the two populations. I can do this for each feature and then rank test points by summing the log likelihoods for each feature. I imagine this could theoretically work well if the smoothing is chosen using cross-validation and there is enough "weirdness" in the distributions that simpler linear classifiers don't capture. 
Edit: I came to the realization that this is essentially Naive Bayes with non-parametric features.

For reference: https://stephens999.github.io/fiveMinuteStats/likelihood_ratio_simple_continuous_data.html
