# How to properly train Faster RCNN

I'm having trouble understanding something when going over Faster R-CNN paper (https://arxiv.org/pdf/1506.01497.pdf).

From what I understood, if k is the number of anchors per position on the feature map, RPN outputs a 4*k vector (for BB regression) and a 2*k vector (for obj/not obj classification) for each position on the feature map (so if i understood well the actual output is a 14x14x4*k and 14x14x2*k).

In the 3.1.3 Training RPNs section, it's said "we randomly sample 256 anchors in an image to compute the loss function of a mini-batch".

So my question is what are we suppose to do with anchors not used during training ? How not to take them into account while there is a value expected for them (class and reg)?

## 1 Answer

Just don't include them as part of the loss function -- since the 256 anchors are sampled randomly, eventually those anchors will probably show up later in the training.