0
$\begingroup$

Let's suppose the following toy example: we are given the task of estimating how many years a person has yet to leave.

For this problem we have tabular data such as age, height, ethnicity, etc; and we also have various pictures from a given person. Furthermore, let's suppose I trained all tabular information into a random forest and got a decent result, I also trained a neural network on the dataset but got worst results than with the random forest.

Considering that we have, say, from 2 to 10 pictures of the person, I am looking for a way to boost the prediction from the random forest considering the pictures for a given person, e.g., training a convolutional NN.

Assuming that the pictures bear information about our final task, I now have two models: one for the pictures and another one for the tabular data. Since a given person can have more more than one picture,

  1. How would I go about training a convolutional NN that considers various samples from a person and returns an averaged prediction over the pictures.
  2. How can I combine my random forest with my newly created ConvNN?
$\endgroup$
1
$\begingroup$

Train a regular classification network but for regression. When you introduce fully-connected layers have the tabular features concatenated to one of them. Concatenated them to the one with fewer features like 50-100 or else they may not be given that much importance due to the presence of too many features.

Alternatively, if you want random forest features, you could concatenate the output of the forest to the last layer or concatenate its features into the neural network.

If you want to use the image features in the random forest, you would have to use an auto-encoder to compress the representation to a small number and use those as features for your forest.This is called representation learning.It was done here (for tabular) and it won the Kaggle challenge:https://www.kaggle.com/c/porto-seguro-safe-driver-prediction/discussion/44629#latest-532540 .

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.