# Do neural networks capture relationships between features?

Say we have a dataset that has several continuous features, 2 time features, and binary classes. I want to be clear, this isn't a time series dataset, but time snapshots of when each observation occurred is recorded. The dataset looks like this:

class    t1    t2    x1      x2      x3      x4      ...
0        735   311   0.345   0.123   -0.32   132
1        93    101   0.89    1.34    9.345   834
...      ...   ...   ...     ...     ...     ...     ...


with t1,t2 as time features and x1,...xn as the continuous features.

Uncaptured explicitly in the dataset, is the difference between times t1 and t2 (t1-t2). This metric is important because if the absolute difference t1-t2 is less than 20, the class will be 1 more than 90% of the time. In other words, the constructed feature t1-t2 is a very good indicator of an observations class.

Will a neural network classifier trained on the original, unchanged dataset above, capture the relationship between t1 and t2 to increase model accuracy?

More broadly speaking, do neural networks capture relationships between features?

I appreciate that in this hypothetical case, if I knew apriori about the time difference indicator, I would engineer a new feature that would be the absolute difference in time, and train the network on a dataset with this new feature. But for the sake of argument, say I do not know this in advance.

• Yes, neural nets finds the features that help to discriminate the classes. This is one of the main selling points of the approach: you can have a dataset with difficult to come up with all necessary heuristics/rules to carry out the classification task. A good example where neural nets have shown great success is image, video and time series data. Those are the areas where before researchers created features manually (sort of) or devised sophisticated techniques to guide discovery of those rules. The downside is that the neural net will not report a list of human readable rules. Jan 19, 2018 at 17:28

Assuming you have enough number of samples in your trainign data, neural networks can learn to capture such uncaptured relationship between features.

This is one of the advantages of using an neural network instead of an tradition ML algorithms like regression, decision trees etc(which needs manual feature extraction).

Since neural networks are like a black box, you cannot be very sure that which feature (or) relation between features is influencing the predictions.

• Do you have any sources / papers / proof for this? Jan 31, 2018 at 8:07
• There a many papers with respect to CNN's whose intermediate layers outputs can be visualized. cs.nyu.edu/~fergus/papers/zeilerECCV2014.pdf
– Avis
Jan 31, 2018 at 8:14
• I made experiments on data where interactions between multiple features were crucial to task resolution. Shallow one-hidden-layer NN did find all relevant interactions between features. Jan 31, 2018 at 8:17

A very interesting question indeed. Where the neural network which task is to f(X) => y would indeed disclose the relationship which help inferring y, not all architectures do have that e.g. autoencoders which in essence do f(X) => X. In case or Autoencoders the optimization problem would found the solution which help finding the more compact representations, rather then helping inferring relationships.