Identify region of interest in image

Question

I'm currently trying to work on the challenge https://www.kaggle.com/c/noaa-right-whale-recognition; I've done basic image recognition work before (Identifying plankton), but this particular challenge requires an additional step: identifying where exactly the whale is in the given photograph, so as to bring the photo size down to a manageable chunk.

I've gone through and drawn rectangular annotations around the whales in the images; however, I'm not clear on how to proceed from here (or whether this was a useful step), and am new to the area, so I don't even have adequate knowledge of the proper terminology to do useful searches.

This appears to be what I gather is referred to as a segmentation problem, but that literature seems to be almost entirely unsupervised, and it's not clear how I would identify and extract the correct segment. I would appreciate pointers to literature/tutorials in the area, especially ones that address:

1. Variable input image sizes
2. Utilizing existing supervised information.

Answer 1

If the images are as clean as that one in the homepage, you can try a simple saliency detection algorithm like this, whose code can be found here. After detecting the saliency, you can write a simple program to draw a rectangle that comprises all the "important" (i.e. non-black) pixels found by the saliency algorithm.

If the images get more complicated, you can try the Efficient Subwindow Search (ESS). It's a little bit more challenging but, if correctly used, it gives you the perfect rectangle automatically. This paper used ESS for ROI detection.

Both algorithms work for variable input sizes. As for using previous information, the first one doesn't use at all; the second one, depending on the implementation, will probably use.