I need to detect persons in a scene.
- I have a 16 bits depth image of that scene (640, 480)
- I have a 16 bits thermal image (80, 60) of the same scene (slighly different point of view)
I resize these two images to (300, 300) for the SSD input. I label on the thermal images since it is easier to see the silhouette.
(Note that I can stack the 2 channels because I know the rotation/translation matrix between those 2 images)
Can I use classic object detection models like SSD or Faster-R-CNN by stacking those images into one input (2, 300, 300) ?
If yes, how should I standardize each channel ?
Or, Is a multimodal architecture, taking 2 distinct inputs and then stacking features, a better way to solve my pb ? Can the multimodal architecture automatically handle the different points of view of my 2 input images ? (I'm not familiar with this second approach, so that's why I ask)
Thermal image :
Depth image :