0
$\begingroup$

I think I understand how MASE works when I have a single time series. But what if I have several, for which I want to obtain an overall accuracy measure?

It's straightforward to compute an aggregate statistic for other measures (MAPE, MAE, RMSE, etc), since there are only 2 vectors of numbers to consider: the predicted values and the actuals. But with MASE, you also need to take into consideration how to compute the scaling factor, which requires a training set.

Code that uses Rob Hyndman's Tidyverts framework would also be great to see, but not essential.

$\endgroup$
1
$\begingroup$

The standard approach is to calculate the MASE separately for each series, using that series' scaling factor (classically, the in-sample MAE from the random walk forecast). Then aggregate these MASEs across series, e.g., by again taking the mean, or the median, or even weighting summary measures by the importance of your series.

$\endgroup$
2
  • $\begingroup$ Ta. Perhaps the default should be to weight by the length of each series? That would mimic the behaviour for the other measures. $\endgroup$ – Hong Ooi May 27 '20 at 7:51
  • $\begingroup$ It depends on what "the behavior for the other measures" is. For the MAPE, for instance, one would usually do the same as I outlined here, taking MAPEs separately per series and then averaging the MAPEs (possibly with some weighting, but that's uncommon). $\endgroup$ – Stephan Kolassa May 27 '20 at 7:56
0
$\begingroup$

A bit late to the party, but my answer could be useful for anyone else dropping to this question at a later stage.

What I'd do is plot the different MASE's in the form of a boxplot. In that way, you prevent taking another mean/median which reduces interpretability and simultaneously allow for easy graphical comparison (compare means, but also spreads and possible outliers).

Disclaimer: I'm absolutely not an expert in this field, merely sharing my personal approach.

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.