# Questions tagged [mase]

The Mean Absolute Scaled Error (MASE) was proposed by Hyndman & Koehler (2006, *International Journal of Forecasting*) as a scale free accuracy measure for point forecasts. It is defined as the ratio of the MAE to the one-step MAE achieved *in-sample* (this is frequently gotten wrong) by a simple benchmark method (often the naive random walk forecast).

38 questions
Filter by
Sorted by
Tagged with
14 views

### What social/economic/political domains are typ. easier-harder to forecast ? Are there benchmark estimates of accuracy for such time series forecasts?

For the report of the results from the Behavioral and Social Science Forecasting Collaborative Tournament I organized during the first COVID year, our team is trying to identify good benchmarks for ...
91 views

### Model performance in time-series forecasting with some outliers

I'm creating forecasts for products where some of them have large seasonal spike during times like Christmas and/or Easter but relatively low sales volume on other times. For this particular product ...
145 views

### Did I calculate the non-seasonal Mean Absolute Scaled Error (MASE) correctly?

This is the formula: Here is the link to page in the book. I am not confident I am interpreting the formula correctly. Below is my data: I calculate the non-seasonal MASE to be 1.125. Is this ...
1 vote
101 views

### Interpretation of scaled error measures

can someone give me an explanation on how one would interpret the result of a scaled error measure. For example the Mean Absolute Scaled Measure (MASE). The numerator is the mean absolute error and ...
• 31
118 views

### Out of sample MASE

When calculating the MASE, the original paper suggests using the in-sample naive forecast error for scaling of the out of sample forecast error. When i use the the MAE generated by a naive forecast on ...
• 23
1 vote
754 views

### Is there any standard / criteria of good forecast measured by SMAPE and MASE?

I have built a forecasting model for a company. Since it is dedicated to practical usage, I prefer to use the relative error parameter (like MAPE, SMAPE, & MASE) as a measurement for my model ...
59 views

### Computing aggregated MASE for multiple time series

I think I understand how MASE works when I have a single time series. But what if I have several, for which I want to obtain an overall accuracy measure? It's straightforward to compute an aggregate ...
• 7,699
81 views

### R - How to create a forecast object from a pre-defined forecast, in order to apply accuracy() for MASE result

I am working with a forecast which has been created by Prophet. I would like to apply accuracy() to return the MASE after identifying this as the best accuracy measure for multiple forecasts, and will ...
80 views

### Do you clean the data before calculating MASE (Mean Absolute Scaled Error)

The denominator in the MASE calculation for seasonal data is the MAE of the seasonal naive forecast calculated in-sample. Is it common to do imputation before calculating the seasonal naive MAE or ...
• 943
6k views

### How do I decide when to use MAPE, SMAPE and MASE for time series analysis on stock forecasting

My task is to forecast future 1 month stock required for retail store, at a daily basis. How do I decide whether MAPE, SMAPE and MASE is a good metrics for the scenario? In my context, over-forecast ...
• 957
1 vote
396 views

### Understanding MASE Value

I've looked through many of the other posts concerning the Mean Absolute Scaled Error (MASE) forecast metric and haven't been able to sort out my problem just yet. I'm working with some weather ...
728 views

### Mean Absolute Scaled Error implementation on multistep time series forecast

The formula for MASE can be found here: https://en.wikipedia.org/wiki/Mean_absolute_scaled_error I am building a multi-step time series forecaster and I want to use MASE as a measure of prediction ...
• 207
1 vote
3k views

### How can MASE (Mean Absolute Scaled Error) score value be interpreted for non time series data?

If I have used MASE to calculate non time-series data error (as described by Dr. Rob Hyndman here), how can I know if the score received is good or not? Since it is not a time-series, a random walk ...
• 11
1 vote
472 views

### MASE and handling nan-values

I'd like to ask advice on how to correctly compute Mean Absolute Scaled Error (2006, Hyndman, Rob J., and Anne B. Koehler.) over the following example: ...
• 35
43 views

### Can I roll-up forecasts to analyze against benchmark methods?

There's a software product (call it Main) we use to allocate parts to various locations. The way it works is it sums up the demand history of all locations to the part level. For example, if part A ...
• 239
27 views

### Is it statistically valid to compare error measures for different sized samples?

I have forecasts for different sized samples using a variety of methods like DES (Double Exponential Smoothing), SES, MA and WA (Weighted Average) to test the strength of the forecasting models. The ...
• 239
1k views

### why is the MSE error higher than MASE and MAPE?

I have a product price time series when I apply two models on them, I calculate all of MSE (Mean Squared Error), MASE (Mean Absolute Scaled Error), and MAPE (Mean Absolute Percentage Error). I noticed ...
• 99
1k views

I have 12 months of in-sample data and 12 months of out-of-sample data. I'm trying to calculate the scaled error for an h-step ahead forecast where h=1, 6 and 12. Do I just calculate the error at ...
• 239
47 views

### Can MASE be calculated for a single horizon but for multiple series?

I have 600 series and I want to calculate the MASE for horizons 1, 3, 6, 8, 10 and 12. I've seen the work of Nikolaos Kourentzes where he calculates an ASE (no Mean) for each of three time horizons ...
• 239
44 views

### Why am I getting different responses from MASE calculations and normality of residuals?

My company uses an inventory model (call it IM) to order parts. I have two years of data (Jan 2016 to Dec 2017) that show how many parts were actually ordered and as well as a forecast from Jan-Dec ...
• 239
75 views

### Is a MASE calculation meaningful if the Test Set is all zeros?

I'm dealing with a dataset of intermittent demand (orders) for the inventory of over 6000 parts. I've extracted one of the parts just to show the type of output I'm getting for a MASE calculation. ...
• 239
1 vote
193 views

### Can MASE be used with a rolling forecast?

I have four years of monthly data, but I want to use three years to test the forecast of a model which uses only one year of historical data in a rolling way. For example: historical data from Jan-...
• 239
870 views

### Is MASE specified only to time series data

Will it be correct to use Mean absolute scaled error in non time series data? I've got a set which contains a lot of zeros, so errors like MAPE can not be used here. MASE based on difference between ...
• 113
740 views

### Multi-input, multi-output time series regression loss using MASE

I have a time series regression that, given a set of lagged values, predicts all of the values from one step ahead to a given horizon. I'm trying to calculate the MASE for this, but am having a ...
• 145
296 views

### Why is MASE scaled by the mean absolute error produced by a naive forecast calculated on the in-sample data

Wouldn't a better scaling factor be with the MAE produced by a naive forecast on the test data itself? When evaluating MASE for the training set, this essentially becomes a comparison for the ...
• 343
2k views

### How to interpret MASE for longer horizon forecasts?

After looking at Hyndman and Koehler, 2006 and applying the metric to my own data, I have been convinced that MASE is a better metric for evaluating forecast error than the method I had been ...
• 1,082
3k views

### Can I use mean absolute scaled error (MASE) from the accuracy function for time series cross validation?

I am using the "forecast" package in R to forecast time series data. I am programming some time series cross validation based off of reading resources from Rob J Hyndman. The last paragraph on page 7 ...
• 283
1 vote
1k views

### Is MASE / Mean Absolute Scaled Error and the usually described denominator appropriate in this case?

I have read through: http://robjhyndman.com/hyndsight/smape/ and https://www.otexts.org/fpp/2/5 and a lot of Hyndman, R. J. and Koehler, A. B. (2006) ‘Another look at measures of forecast accuracy’, ...
• 215
1 vote
979 views

### Calculate MASE for time series with multiple seasonalities

What is an appropriate way to calculate the MASE accuracy measure (Link) for a time series with multiple seasonalities? For example: daily data with a strong weekly pattern and annual pattern. ...
• 485
1 vote
165 views

### Time series data pre processing - model improves?

I have a time series that includes some rare extreme values. We are talking about daily data, in total 1461 observations and 11 extreme values. I adjusted those 11 values with a multiple regression. ...
• 485
4k views

### Acceptable limit for MASE

What are good sign of fit from result of forecast::accuracy. How to interpret ...
• 131
1 vote
462 views

### Calculating MASE Scores for Individual Participants

I have run a controlled experiment in which 30 participants made one step ahead forecasts under two separate conditions. I want to know if there is a difference in error scores between the two ...
18k views

### ARIMA: How to interpret MAPE?

I am using the forecast package in R to generate an ARIMA model for my data. I started with the auto.arima function for a try and got a ARIMA(1,1,2) model. ...
• 1,119
141 views

### Mean Absolute Scaled Error [duplicate]

Right now, I am analyzing the prediction quality of a dynamic model that has variables with different units (e.g. $x_{1,t}$ is in meters, $x_{2,t}$ is in kilograms etc.). I have discovered a great ...
• 2,483
37k views

### Interpretation of mean absolute scaled error (MASE)

Mean absolute scaled error (MASE) is a measure of forecast accuracy proposed by Koehler & Hyndman (2006). $$MASE=\frac{MAE}{MAE_{in-sample, \, naive}}$$ where $MAE$ is the mean absolute error ...
• 55.5k
12k views

### Time series forecasting accuracy measures: MAPE and MASE

We come to this toy example showing MAPE and MASE are not consistent when measuring forecasting accuracy. Data consist of 100 white noise and 100 $AR(1)$ time series with length $N=500$, mean $\mu=1$ ...
• 417
1 vote
1k views

### Alternative to MAPE when the data is not a time series

I have a data set where many of the actual values are zero, so I can't use MAPE. It's not a time series, so I can't use MASE ala our very own Rob Hyndman. Is there another alternative to MAPE that I ...
• 1,047
1 vote