Iterating over set of functions in logistic regression

Question

I am trying to find the best fitting function in case of logistic regression using R.

regr_A<-glm( cbind(X$M, X$N-X$M)~...., data=mydata, family="binomial"

I want to have the best "composition" of only three regressors

for example like

regr_A<-glm( cbind(X$M, X$N-X$M)~exp(v1^2)+sin(v2)+v3^8, data=mydata, family="binomial"

but also maybe the best fit

regr_A<-glm( cbind(X$M, X$N-X$M)~cos(v2)+v3^3, data=mydata, family="binomial"

whatever.

So is a way to find a function f = g(v1,v2,v3), where g can be everything (the composition of whatever : polynomials, sin, cos etc), that the glm is fitting best?

Sure one can look at the scatter plot and "get a feeling" for the objective function, but is there a way to find it automatically? like to test many possibilities?

I am aware that it is not possible to test an infinite number of functions - but do you see a practical possibility ?

something like this:

regr_A <- glm(cbind(X$M, X$N-X$M) ~ HERE IS A LIKELY COMBINATION OF POLYNOME FUNCTIONS, EXP FUNCTIONS etc), data=mydata, family=binomial)

So a minimization problem (for example via deviance or AIC).I just don't know the best way to iterate over the function set?

So there are three regressors

x,y,z and for example polynomials up to degree 10 (of all possible combinations of the three variables) are to be tested and the fitting function which has the smallest AIC is to be chosen.

Answer 1

There are two approaches to building models:

• In some cases you know what the functional form should be, so you use this knowledge when building the model. This is often the case in statistics. This may be the case, for example, when there is a known physical relationship between the variables. Another example might be time-series forecasting, where there are some "usual suspects" for feature engineering like Fourier-transformations.

• Another possibility is when you don't know the functional relationship. This is usually the case of using machine learning. In such a scenario, you need a model that is flexible enough to find the appropriate functional form. It can be achieved by using polynomials because they can approximate any function. There are also many models that are universal approximators like kernel regression, tree-based models, $k$NN, neural networks, etc. We use those models because we can't "try all the possible functions" (the infinite search space) while the models by themselves are able to approximate any function.

Finally, if you want to "try all the possible combinations and pick the best one" you are likely to end up with an overfitting model, so you need additional safeguards to prevent this.