2
$\begingroup$

Say we have a dataset that has several continuous features, 2 time features, and binary classes. I want to be clear, this isn't a time series dataset, but time snapshots of when each observation occurred is recorded. The dataset looks like this:

class    t1    t2    x1      x2      x3      x4      ...
0        735   311   0.345   0.123   -0.32   132
1        93    101   0.89    1.34    9.345   834
...      ...   ...   ...     ...     ...     ...     ...

with t1,t2 as time features and x1,...xn as the continuous features.

Uncaptured explicitly in the dataset, is the difference between times t1 and t2 (t1-t2). This metric is important because if the absolute difference t1-t2 is less than 20, the class will be 1 more than 90% of the time. In other words, the constructed feature t1-t2 is a very good indicator of an observations class.

Will a neural network classifier trained on the original, unchanged dataset above, capture the relationship between t1 and t2 to increase model accuracy?

More broadly speaking, do neural networks capture relationships between features?

I appreciate that in this hypothetical case, if I knew apriori about the time difference indicator, I would engineer a new feature that would be the absolute difference in time, and train the network on a dataset with this new feature. But for the sake of argument, say I do not know this in advance.

Improve this question
$\endgroup$
1
  • $\begingroup$ Yes, neural nets finds the features that help to discriminate the classes. This is one of the main selling points of the approach: you can have a dataset with difficult to come up with all necessary heuristics/rules to carry out the classification task. A good example where neural nets have shown great success is image, video and time series data. Those are the areas where before researchers created features manually (sort of) or devised sophisticated techniques to guide discovery of those rules. The downside is that the neural net will not report a list of human readable rules. $\endgroup$
    – Vladislavs Dovgalecs
    Jan 19, 2018 at 17:28

2 Answers 2

Reset to default
3
$\begingroup$

Assuming you have enough number of samples in your trainign data, neural networks can learn to capture such uncaptured relationship between features.

This is one of the advantages of using an neural network instead of an tradition ML algorithms like regression, decision trees etc(which needs manual feature extraction).

Since neural networks are like a black box, you cannot be very sure that which feature (or) relation between features is influencing the predictions.

Improve this answer
$\endgroup$
3
  • $\begingroup$ Do you have any sources / papers / proof for this? $\endgroup$
    – PyRsquared
    Jan 31, 2018 at 8:07
  • 1
    $\begingroup$ There a many papers with respect to CNN's whose intermediate layers outputs can be visualized. cs.nyu.edu/~fergus/papers/zeilerECCV2014.pdf $\endgroup$
    – Avis
    Jan 31, 2018 at 8:14
  • 1
    $\begingroup$ I made experiments on data where interactions between multiple features were crucial to task resolution. Shallow one-hidden-layer NN did find all relevant interactions between features. $\endgroup$
    – Alexey Burnakov
    Jan 31, 2018 at 8:17
0
$\begingroup$

A very interesting question indeed. Where the neural network which task is to f(X) => y would indeed disclose the relationship which help inferring y, not all architectures do have that e.g. autoencoders which in essence do f(X) => X. In case or Autoencoders the optimization problem would found the solution which help finding the more compact representations, rather then helping inferring relationships.

Improve this answer
$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.