0
$\begingroup$

I would like to try extracting "filters" (similar to the facial feature extraction) from a multi-task three hidden-layer neural network with ReLU activations. My data is a 1-D vector of counts and I'd like to extract higher level features. However, I'm having trouble finding any resources on feature extraction from non-convolutional neural networks beyond the first layer. Is this a thing that makes sense and does anybody know any papers or know a formula I could implement to find a ranking of bin importance for a particular node in a hidden layer?

Things I've been thinking of using, but am not sure if I'm on the right track:

  • gradient ascent from random perturbation of the input vector
  • looking at inputs that activate the data the best and then masking out data
Improve this question
$\endgroup$
1
$\begingroup$

There is few/nothing written on feature extraction for 1D using DNN's. This is mostly because the idea of extracting visual features from a image and then using this as an input to a classifier was(is?) the standard way of doing ML for computer vision. And since the rise of CNN's people showed that the hidden convolutional layers do a similar job to the feature extractors previously used, and this is one the reasons it works. This parallel between 1D data feature extractors and hidden layers of NN's was never deeply investigated though people knew it generated some kind of feature.

Yet, the interpretation of features from CNN's is easily adapted to 1D DNN's by investigating the outputs of each hidden layer as a "feature" or "feature of feature" the higher you go into the NN layer level.

Improve this answer
$\endgroup$
0
$\begingroup$

autoencoders can be used to learn features from an input vector. one more more layers can be used to learn a mapping between the input vector and itself. Thereby learning new ways(features) to represent the original information. There is a substantial amount of literature on autoencoders that you could easily find if that suits your need. This link shows how to extract features using autoencoders http://ufldl.stanford.edu/tutorial/unsupervised/Autoencoders/

Improve this answer
$\endgroup$
1
  • $\begingroup$ It may be clear to you how autoencoders can help the OP but elaborating a bit more on how they can address their question would be of even more help. Another thing would be to cite an article or two that directly answer the OPs query. $\endgroup$
    – Mike Hunter
    Mar 28 '16 at 17:18

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.