I am fairly new to neural networks. I am trying to empirically show that a neural network can work better than logistic regression when the underlying function is non-linear. In my simulation study, the true probability of assignment to the treatment group is a function of $X_1$, $X_2^2$, $X_3$, and $X_4$ - however, I am acting as if I don't know that $X_2$ is squared, and just using the non-squared term. Logistic regression performs poorly.
I think that I can get a neural network to estimate the function better since it's a (rather simple) non-linear function. However, my first attempt was not fruitful and I'm not sure if I need to increase "hidden," the number of hidden neurons, or change the activation function (or something else). Here's what I tried in R:
nn = neuralnet(t ~ X1+X2+X3+X4, data=df, hidden=3, act.fct = "logistic", linear.output = FALSE)