Firstly, thanks for posting a very interesting question.
To answer it shortly, a vanilla convnet trained end-2-end to predict age from a photo will be generally prone to mis-classify images such as the one you posted. Secondly, note that accurately estimating the age of a person is a nearly impossible task1.
The main difference from your proposed approach using some object detectors (be it RCNN, Faster RCNN, YOLO or SSD) is that you are using different information to train the models. The CNN is trained only on images and needs to find out all the necessary features itself. It is most likely going to find various facial features, but it will also rely on clothing and perhaps scene features (kids may be often in the picture with some toys, adults will be more likely in office environments, etc.). These features will not be robust to your counterexample.
On the other hand, if you train the network to explicitly detect objects as "torso" and "head", you are providing extra information that these objects are important for the task, and thus simplify the problem2.
While the approach of detecting head and torso and then evaluation the size ratio of the bounding boxes sounds interesting, I can see several obstacles:
- Obtaining data: I am not aware of the availability of large dataset where both age and bounding boxes would be present.
- Imperfect FOV: In most images (e.g. both your examples), the people are not displayed whole. You would have to deal with the fact that the torso bounding boxes would not be always perfect simply because part of the person is not in the image and the net would have to guess how large part is missing (and the ground truth bounding boxes would most likely not capture this information). Also, the aforementioned object detectors don't always handle predictions of partial objects properly. This might introduce too much noise in the model.
- Various poses: The torso-to-head ratio would be very different for people viewed frontally and from the side.
- Adults: It seems the ratio works well to predict ages between 0-21, but I don't see how it would help to predict ages of adults (I suppose the ratio does not change in higher age).
All these problems suggest that the head-to-torso ratio approach is also not going to work perfectly, although it might be more robust to your particular counterexample.
I guess the best way to perform this task would be to 1) detect the face, 2) predict age only from the facial crop (removes potentially misleading information). Note that some R-CNN-like architecture using ROI-pooling could be trained to do this end-2-end.
1 Even using very sophisticated medical methods (which are arguably much more informative than a photo of the person) this is not possible to do accurately. See this Quora thread for more information.
2 Check the article Knowledge Matters: Importance of Prior Information for Optimization for an example how providing some intermediate knowledge about the task can greatly simplify learning.