Return to Answer

Cons: Overestimates and underestimates are not penalized equally; you have to divide by zero if a true value is zero; many others, as described on the Wikipedia article on MAPE

Cons: Overestimates and underestimates are not penalized equally; you have to divide by zero if a true value is zero; many others, as described on the Wikipedia article on MAPE, though the link also mentions some alternatives

(This is a topic for a separate question (maybe one I will post with a self-answer that I could link back here), but I disagree with the exact implementation of out-of-sample $R^2$ in the common Python machine learning package sklearn. That implementation compares your performance to a model that always guesses the out-of-sample mean, which is supposed to be a model that you cannot access (since the out-of-sample data are not for training).)

Cons: Overestimates and underestimates are not penalized equally, you have to divide by zero if a true value is zero, many others, as described on the Wikipedia article on MAPE

(For reasons I discuss in detail here, I disagree with the exact implementation of out-of-sample $R^2$ in the common Python machine learning package sklearn. That implementation compares your performance to a model that always guesses the out-of-sample mean, which is supposed to be a model that you cannot access (since the out-of-sample data are not for training).)

Cons: Overestimates and underestimates are not penalized equally; you have to divide by zero if a true value is zero; many others, as described on the Wikipedia article on MAPE

Cons: The relationship to variance can range from unhelpful to downright misleading if the error is not Gaussian or does not have a constant variance

Pros: Handles data on different scales, where missing by $5$ might be a big deal when the true value is $10$ but less of a big deal when the true value is a billion

Cons: Overestimates and underestimates are not penalized equally, you have to divide by zero if a true value is zero, many others, as described on the Wikipedia article on MAPE

Cons: The relationship to variance can range from unhelpful to downright misleading if the error is not Gaussian or does not have a constant variance; the units are squared

$RMSE: \text{Root Mean Squared Error}$

(This is just the square root of the MSE.)

Pros: Related to the standard deviation of the error term; easy to calculate; in the same units of $y$

Cons: The relationship to standard deviation can range from unhelpful to downright misleading if the error is not Gaussian or does not have a constant standard deviation

Pros: Handles data on different scales, where missing by $5$ might be a big deal when the true value is $10$ but less of a big deal when the true value is a billion

Cons: Overestimates and underestimates are not penalized equally, you have to divide by zero if a true value is zero, many others, as described on the Wikipedia article on MAPE