It is a different thing to fit matrix A to matrix B and to fit matrix B to matrix A, just like it is a different thing to regress y on x and regress x on y – although all these tasks are related.
As to your question:
1a. In symmetric Procrustes, the goodness-of-fit statistics are identical in B to A and A to B. However, many other things differ (see https://stackoverflow.com/questions/42335149/interpreting-procrustes-output-in-vegan/42352087#42352087).
1b. I wrote procrustes() function some 20 years ago to fit matrix B to target matrix A: I wanted to get the rotated/scaled/translated matrix B that is similar to A. If that is your goal, you should use non-symmetric (B to A) rotation. If you do not want to get the result (rotation), but you are just interested in the similarity between two matrices and the statistic for that similarity, you should use symmetric rotation. Non-symmetric analysis is analogous to regression analysis (B on A), and symmetric Procrustes is analogous to correlation analysis (similarity between A & B without considering direction).
1c. This depends on your study question (that I do not know). Do you want to have a "regression" or a "correlation"?
- The authors of Protest (see references in the software help page) suggested the method with symmetric Procrustes, and the implementation follows the original paper (Peres-Neto & Jackson 2001, Oecologia 129: 169-178).
2a. I guess you can – in principle – but please read the paper: they may have an argument against non-symmetric analysis. However, you cannot run that in vegan, because it is not implemented there. Without re-reading the paper and considering the issue, I think that the reason is just that Peres-Neto & Jackson were only interested in non-directional (correlation-like) comparison of matrices and therefore only considered symmetric analysis. My gut-feeling is that non-symmetric analysis would give different numeric indicators but the same permutation P-values (just like tests on the slope of regression and on correlation coefficient give similar results). You may edit the function to see if this is the case (I have no interest of doing so).