Let's suppose the following toy example: we are given the task of estimating how many years a person has yet to leave.

For this problem we have tabular data such as age, height, ethnicity, etc; and we also have various pictures from a given person. Furthermore, let's suppose I trained all tabular information into a random forest and got a decent result, I also trained a neural network on the dataset but got worst results than with the random forest.

Considering that we have, say, from 2 to 10 pictures of the person, I am looking for a way to boost the prediction from the random forest considering the pictures for a given person, e.g., training a convolutional NN.

Assuming that the pictures bear information about our final task, I now have two models: one for the pictures and another one for the tabular data. Since a given person can have more more than one picture,

  1. How would I go about training a convolutional NN that considers various samples from a person and returns an averaged prediction over the pictures.
  2. How can I combine my random forest with my newly created ConvNN?
  • $\begingroup$ How did you end up solving this? $\endgroup$ – BLBA Apr 21 '20 at 15:12
  • $\begingroup$ @AnnieTheKatsu We tried to concatenate the output of the random forest to an input to the neural net, but it did not result in a better model. Instead, we decided to train a classifier for one of the inputs using a neural net and then use the average estimated output of the neural net to shift the estimation of the random forest. $\endgroup$ – Gerardo Durán Martín Apr 28 '20 at 11:43
  • $\begingroup$ That's iteresting. If I may ask your random forests produced regression outputs or did were they somehow used for classification (before concatenation) $\endgroup$ – BLBA Apr 28 '20 at 13:14

Train a regular classification network but for regression. When you introduce fully-connected layers have the tabular features concatenated to one of them. Concatenated them to the one with fewer features like 50-100 or else they may not be given that much importance due to the presence of too many features.

Alternatively, if you want random forest features, you could concatenate the output of the forest to the last layer or concatenate its features into the neural network.

If you want to use the image features in the random forest, you would have to use an auto-encoder to compress the representation to a small number and use those as features for your forest.This is called representation learning.It was done here (for tabular) and it won the Kaggle challenge:https://www.kaggle.com/c/porto-seguro-safe-driver-prediction/discussion/44629#latest-532540 .


Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.