I want to build a regression NN model that predicts Y variable but takes into account the interactions between input variables (x1,x2,x3,x4,x5) without explicitly specifying them. My current NN model implementation has:

  • 5 input variables (x1,..x5)
  • 1 hidden layer with 5 neurons
  • 1 output (Y)
  • logistic activation function

My question is: Can this architecture capture the interactions (if exist) or do I need an extra layer for this?

Thank you!


Think of a neural network as an additive model: Without the hidden layer, you already achieve the same as a regression model without interactions, as the network can learn the weight of each of the individual inputs, which is just like estimating the coefficients of the input variables in a regression model.

A single hidden layer can learn how simple combinations of the input relate to the output. Think of this as a regression model with first-order interactions (except that it is not limited to linear combinations). So yes, your model should already be able to capture interactions. However, with only 5 nodes in the hidden layer, it may not capture many different combinations, if there are indeed many different interactions that affect the outcome. Whether this is likely is highly dependent on what kind of data these are and what kind of outcome you are predicting.

Adding more layers would allow the network to make further abstractions from those made in the first hidden layer. This is analogous to higher order interactions in a regression model.

  • $\begingroup$ Perfect answer.. I was editing an answer saying the same thing about regression similarity $\endgroup$ – Fr1 Aug 11 at 11:17
  • $\begingroup$ Thank you so much @Frans Rodenburg! Helps me so much in my current project :) $\endgroup$ – nba2020 Aug 12 at 7:43
  • 1
    $\begingroup$ @nba2020. You're welcome! Good questions are at least as valuable as good answers. $\endgroup$ – Frans Rodenburg Aug 13 at 13:48

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.