What is "symmetry" in evaluation metrics

Question

I'm seeing Mean absolute percentage error (MAPE) is not symmetric. Tried to understand what is symmetry here but didn't find a good answer online.

Can I ask:

• What is symmetry in evaluation metrics? If you can give an easy to understand example will help a lot
• Why MAPE is not symmetric?
• What's the pros and cons for an evaluation metric being symmetric

Answer 1

Consider an evaluation metric $e(\hat{y},y)$ that takes a forecast $\hat{y}$ and an actual $y$.

• "Symmetry" has various definitions. Often they are not stated explicitly. For instance:

  • We could call $e$ "symmetric" if the result is the same if we exchange the forecast and the actual: $$e(\hat{y},y)=e(y,\hat{y}).$$ That is, we have the same error if we forecast $5$ when the actual is $3$ and vice versa.
  • Or we could call $e$ "symmetric" if over- and underforecasting by the same amount yields the same error: $$e(y+\delta,y)=e(y-\delta,y).$$

• The MAPE is not symmetric in the first sense, because $\text{MAPE}(5,3)=67\%$, while $\text{MAPE}(3,5)=40\%$. The so-called "symmetric MAPE" or sMAPE is symmetric in this sense.

  However, the MAPE is symmetric in the second sense. (And the sMAPE is not, see Goodwin & Lawton, 1999, IJF).

• Symmetry is, in my opinion, mainly about aesthetics. Humans are wired to like symmetry.

  Whether you should require your evaluation metric to be symmetric in one of the two senses above (and in which sense), should be guided by what you are predicting, and by how costly your errors are. I can imagine over- and underforecasts being equally costly, which would argue for a symmetric evaluation metric in the second sense above (so the MAPE, MAE and MSE would quality, but the sMAPE would not). However, I have a harder time arguing for the first kind of symmetry - when would it ever be important that switching the forecast and the actual should give the same result on the evaluation metric?