I have a set of forty predictors each of which are binary. I’m using elastic net logistic regression in addition to a random forest
Is there any reason why you could not use binary inputs for a Keras neural network?
I can’t find anything in the documentation that indicates that this is not an option.
There's no reason you would not be able to apply a neural network (in Keras or any other software) using binary inputs.
Indeed, the XOR problem is a canonical example of how a neural network is useful, even when the inputs are binary. The XOR task is to have a classifier separate 4 binary datapoints into 2 categories:
| $X_1$ | $X_2$ | $Y$ |
|---|---|---|
| 1 | 1 | 1 |
| 1 | -1 | -1 |
| -1 | 1 | -1 |
| -1 | -1 | 1 |
Drawing a single line in $X_1, X_2$ space won't separate the 2 classes of $Y$. However, using interactions among $X_1, X_2$ will. This is achievable with a neural network with a hidden layer.
In other words, interactions among input features, even binary input features, can be essential. Neural networks are one way to achieve that.
You mention logistic regression in your question. XOR can also be solved using logistic regression, if you apply feature-engineering and construct interactions manually. For the XOR problem, it is clear that the $X_1 X_2=Y$; logistic regression will succeed. (Though it will exhibit perfect separation phenomena, unless you take steps to prevent it.)
And obviously a decision tree-based model would have no difficulties with this toy problem. A single tree with two splits would fit this data perfectly.
Nothing statistical keeps you from doing this.
I do not see much value to the nonlinearity that a neural network will introduce. However, a neural network might do a good job of capturing variable interactions, and those interactions could be key. My example here, for instance, gives an example where you need the interaction to distinguish between outcomes.
