5
$\begingroup$

I've just been watching some computerphile videos on ANNs. And in one of them the guy talked about figuring out the price of a house according to a picture.

At that moment an idea came into my head. Maybe this problem could be better solved if it had more information that the picture simply cannot give. For example, area, number of bathrooms, number of bedrooms, neighbourhood info (schools nearby, hospitals, etc).

The information I just gave could also be used to determine the same thing the picture tries to determine but it could be solved using a different type of ANN ("regular" feed forwardd network deep learning techniques). And in my understanding these type of informations are treated differently than the ones obtained from the figure.

My question is: Is it possible to mix the 2 different types of data? For example:

image kernel outputs -> input neurons other information -> other inputs

And then the input data from the kernels and the information is "mixed" in the hidden neurons. Am I dreaming? I don't know practically anything about convolutional neural networks and I only have a vague idea about how the input to the system works (I think I learned it from the computerphile videos, they really explained it like I'm five), but is it possible to do what I'm thinking?

Improve this question
$\endgroup$

1 Answer 1

Reset to default
4
$\begingroup$

Is it possible to mix the 2 different types of data?

Yes, no problem.

You can also have more than one image as input, e.g. Bojarski, Mariusz, Davide Del Testa, Daniel Dworakowski, Bernhard Firner, Beat Flepp, Prasoon Goyal, Lawrence D. Jackel et al. "End to End Learning for Self-Driving Cars." arXiv preprint arXiv:1604.07316 (2016).

enter image description here

Improve this answer
$\endgroup$
3
  • $\begingroup$ Ok. This mixes 3 CNNs and generates an angle output for the steering wheel. This is great, amazing article. But does it mean that I could mix other data like number of passengers, extra weight, wind speed and consumption so that it can steer better, as well as optimize fuel consumption? The type of input data seems to be so different (and the training algorithms look like they need to be different as well) $\endgroup$
    – morcillo
    Commented Jul 17, 2016 at 22:24
  • $\begingroup$ I can't upvote your answer because I have less than 15 reputation $\endgroup$
    – morcillo
    Commented Jul 17, 2016 at 22:25
  • $\begingroup$ @morcillo yes you could mix other data, no issue. Typically those CNN networks end with some feedforward layers, you could plug your other data there. $\endgroup$
    – Franck Dernoncourt
    Commented Jul 17, 2016 at 23:49

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.