If we assume the distribution of hits is binormal, then the distribution of the distance to the center of the target is a scaled Chi distribution with two degrees of freedom. From this (applying its inverse CDF), we find that its 95th percentile coincides with the radius of 16/2 = 8 centimeters when the scaling factor equals 1.33523. This factor is also the standard deviation of the components of the binormal distribution. Integrating the PDF of that binormal distribution over a 13 by 10 cm rectangle centered at the point of aim gives 0.83311, the desired value of $B$.
Here's a picture showing a shaded contour plot of the PDF restricted to that rectangle, with the circular target behind it for reference:

The value of 0.83311 was found with Mathematica:
With[{s = 8 / InverseCDF[ChiDistribution[2], 0.95]},
NIntegrate[PDF[BinormalDistribution[{0,0},{s,s},0],{x,y}], {x,-13/2,13/2}, {y,-10/2,10/2}]
]
It was checked by simulating 100,000 independent shots and reporting the proportions that (a) fell within the 16 cm circular target and (b) fell within the rectangular target:
With[{s = 8 / InverseCDF[g, 0.95], n = 100000},
data = RandomReal[BinormalDistribution[{0, 0}, {s, s}, 0], n];
old = Length[Select[data, Norm[#] <= 8 &]] / n;
new = Length[Select[data, Abs[#[[1]]] <= 13/2 && Abs[#[[2]]] <= 10/2 &]] / n;
{old, new} // N
]
The output of {0.94919, 0.83331} is close enough to the intended values of {0.95, 0.83311} to confirm the correctness of the calculations.