Neural Networks interaction between input variables I want to build a regression NN model that predicts Y variable but takes into account the interactions between input variables (x1,x2,x3,x4,x5) without explicitly specifying them. My current NN model implementation has:


*

*5 input variables (x1,..x5)

*1 hidden layer with 5 neurons

*1 output (Y)

*logistic activation function


My question is: Can this architecture capture the interactions (if exist) or do I need an extra layer for this?
Thank you!
 A: Think of a neural network as an additive model: Without the hidden layer, you already achieve the same as a regression model without interactions, as the network can learn the weight of each of the individual inputs, which is just like estimating the coefficients of the input variables in a regression model.
A single hidden layer can learn how simple combinations of the input relate to the output. Think of this as a regression model with first-order interactions (except that it is not limited to linear combinations). So yes, your model should already be able to capture interactions. However, with only 5 nodes in the hidden layer, it may not capture many different combinations, if there are indeed many different interactions that affect the outcome. Whether this is likely is highly dependent on what kind of data these are and what kind of outcome you are predicting.
Adding more layers would allow the network to make further abstractions from those made in the first hidden layer. This is analogous to higher order interactions in a regression model.
