I am trying to understand the faster RCNN but I can't understand the meaning of anchors' scales? Especially in this article Faster RCNN.
The author considers 3 scales $(128^2, 256^2, 512^2)$
What does this line mean?
I know that for each $3*3$ spatial location in the feature map (VGG) we perform convolution and after that we do conv for each anchor box. the receptive field of those $3*3$ spatial locations are $(16*3)^2$ in the original image and I think that that means the anchors area should be smaller than $(16*3)^2$. Isn't $512^2$ too big for an anchor? What happens when it's near the edge of the original image?