5
$\begingroup$

I want to build a regression NN model that predicts Y variable but takes into account the interactions between input variables (x1,x2,x3,x4,x5) without explicitly specifying them. My current NN model implementation has:

  • 5 input variables (x1,..x5)
  • 1 hidden layer with 5 neurons
  • 1 output (Y)
  • logistic activation function

My question is: Can this architecture capture the interactions (if exist) or do I need an extra layer for this?

Thank you!

$\endgroup$

1 Answer 1

5
$\begingroup$

Think of a neural network as an additive model: Without the hidden layer, you already achieve the same as a regression model without interactions, as the network can learn the weight of each of the individual inputs, which is just like estimating the coefficients of the input variables in a regression model.

A single hidden layer can learn how simple combinations of the input relate to the output. Think of this as a regression model with first-order interactions (except that it is not limited to linear combinations). So yes, your model should already be able to capture interactions. However, with only 5 nodes in the hidden layer, it may not capture many different combinations, if there are indeed many different interactions that affect the outcome. Whether this is likely is highly dependent on what kind of data these are and what kind of outcome you are predicting.

Adding more layers would allow the network to make further abstractions from those made in the first hidden layer. This is analogous to higher order interactions in a regression model.

$\endgroup$
6
  • 1
    $\begingroup$ Perfect answer.. I was editing an answer saying the same thing about regression similarity $\endgroup$
    – Fr1
    Aug 11, 2019 at 11:17
  • $\begingroup$ Thank you so much @Frans Rodenburg! Helps me so much in my current project :) $\endgroup$
    – nba2020
    Aug 12, 2019 at 7:43
  • 1
    $\begingroup$ @nba2020. You're welcome! Good questions are at least as valuable as good answers. $\endgroup$ Aug 13, 2019 at 13:48
  • 1
    $\begingroup$ This is actually completely wrong. As I understand it, additive models cannot capture interaction effects as those require a multiplicative network. $\endgroup$ Dec 7, 2020 at 1:54
  • $\begingroup$ See also: stats.stackexchange.com/questions/279525/… $\endgroup$ Dec 7, 2020 at 2:13

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.