Is MASE specified only to time series data

Will it be correct to use Mean absolute scaled error in non time series data? I've got a set which contains a lot of zeros, so errors like MAPE can not be used here. MASE based on difference between yt and yt-1 (formula like here), but I don't have time series, just observations of some activity of 200 people (independent), so maybe I can just use difference based on second and first person and so on? If not, do you know any percentage measures that I can use in that kind of set?

(The standard version of) MASE only works for time series. The purpose of the $Y_t$ - $Y_{t-1}$ term in the denominator is that it represents a naive forecast, that it is your best guess for $Y_t$ is $Y_{t-1}$.

In your case, your data is non-sequential and more importantly it is independent, so the idea of using $I_{n-1}$ as an estimator for $I_n$ doesn't really make sense (why $I_{n-1}$? Why not $I_{n-2}$ or $I_{n-199}$?)

To answer your 2nd question, it depends on what error you are trying to measures exactly ?

For dealing with the issue of zeros, a possible alternative for MAPE is sMAPE (Symmetric Mean Absolute Percentage Error). That might be a good option for you.

• When $Y_{emp}$ and $Y_{pred}$ are both equal to zero, than you set the sMAPE to 0 (since they are equal to each other, there is no error) (add an if clause to your calculation). Many software packages do that already, even with MAPE.
• For the second problem, you can use the modified version of the sMAPE:

$$\text{sMAPE} = \frac{1}{N}\sum \frac{|y_{emp} -y_{pred}|}{{|y_{emp}|+|y_{pred}|}}$$

This gives you a 100% error in cases like you mentioned ($Y_{emp} = 0$ and $Y_{pred}=0.005$)

So the modified version of sMAPE could be:

$$\text{sMAPE} = \begin{cases} 0, & \text{ if } Y_{emp} = 0 & and & Y_{pred}=0 \\ \frac{1}{N}\sum \frac{|y_{emp} -y_{pred}|}{{|y_{emp}|+|y_{pred}|}} & \text{otherwise} \\ \end{cases}$$

If even 100% is too high a threshold, you can apply some sort of log transformation.

Since I first wrote this answer, I noticed elsewhere in this forum that Rob Hyndman (the original author of MASE) proposes a modified version of MASE for non-time series data. See here

If $e_j$ denotes a prediction error on the test data, then the scaled errors are $$q_{j} = \frace_{j}}{\displaystyle\frac{1}{N}\sum_{i=1}^N |Y_i-\bar{Y}|}.$$ where $y_1,\dots,y_N$ denotes the training data.

Note here that $Y_{t-1}$ is replaced with $\bar{Y}$.

In the time series case, this would mean that the prediction is being compared to the predictions produced by a mean forecast instead of a naive forecast (as it is in the standard formulation of MASE).

• I 've read about this measure but i can't use this because of situation when I have $Y_{emp}$ =0 and $Y_{pred}$ =0, then this measure is bad. Also When I check only freq, when $P_{emp}$ =0 and $P_{pred}$ =0,0005 then this measure going to its top (200%). I need error which can by comparable and don't have problems like I texted above. – Roberto Jan 9 '18 at 17:14
• @Roberto see my edits. – Skander H. Jan 9 '18 at 17:45
• Well that makes sens for probabilities. Thank You for your help. Also I am looking on the last formula in the wikipedia you mentioned and I think it solve a problem to both. Am I right? – Roberto Jan 9 '18 at 18:00
• @Roberto the last formula in the wikipedia page - log(F/A) - requires strictly positive data and you said you have a lot of zeros? Also it would lead to cases where the error is negative for F < A . – Skander H. Jan 9 '18 at 18:13
• I meant SUM(abs[F-A]) / SUM(A+F). And yes, 80% of y in my set equal zero. – Roberto Jan 9 '18 at 18:16