Using Neural Net weights as input to another classifer Is there anyway to use the weights from a neural net hidden layer as input to another classifier, say a random forest? Of course this is trivial for the training data but how to score new data? Are there any methodologies for this type of ensemble?
EDIT #1
Here is a concrete simple example: The training data has 10,000 rows and 2 variables (columns). I want to use the hidden layer of a MLP to do feature engineering - creating new variables from the original 2 that can then be passed into another learning algorithm (along with the original 2 features). If the MLP built has 2 inputs (ignore the bias) and the hidden layer has 2 units, then the weight matrix from input layer to hidden layer would be dimension (2,2) where the entry (1,2) represents the weight from input #1 to hidden unit #2.
I can extract this weight matrix for the training data and pass it to the classifier further down the pipeline. However, my question is how to follow this approach for the test set / new data?
 A: I am not quite sure what you actually plan to classify using that approach. If you use the weights themselves as input to another classifier then you would classify the layer that uses these weights?!
I presume you plan to use the output of the last (or some intermediate) layer of your network in another classifier. In that case you simply pass your testing data through the network, observe the output at the layer you are interested in (i.e. just compute the forward pass) and pass those along to the random forest (or any other classifier) for scoring. This process will not be optimal in the sense of some loss function, though, as you do not perform end-to-end learning (i.e. you do not pass the random forest's loss along down the neural network).
If this is not what you intended then please clarify ;)
A: Wouldn't it make more sense to just use the outputs of your hidden layer as new features? It makes more sense to me to cut off the output layer.
It's important to be aware of the fact that the weights in a neural network behave quite randomly. If you use a different seed for initialization, the weights will converge to different values (in the best case they will be comparable but permuted). There is no implicit order in the weights of hidden nodes even when a clear structure can be retrieved every time (unlikely).
