Skip to main content

Questions tagged [mase]

The Mean Absolute Scaled Error (MASE) was proposed by Hyndman & Koehler (2006, *International Journal of Forecasting*) as a scale free accuracy measure for point forecasts. It is defined as the ratio of the MAE to the one-step MAE achieved *in-sample* (this is frequently gotten wrong) by a simple benchmark method (often the naive random walk forecast).

Filter by
Sorted by
Tagged with
2 votes
1 answer
71 views

Seasonal Time Series Forecasting and MASE

I am creating a a number of forecasts for sales at various levels of data aggregation, based on the properties of the products (eg. is it in a bottle or a can). I plan to create multiple models and ...
ForecastNoob's user avatar
2 votes
1 answer
56 views

Diebold-Mariano test for one-step forecasts using Mean Absolute Scaled Error

This is my first time doing time series forecasting, so I am sorry for any inconsistencies in my question. But I have two different models that I want to compare. On Wikipedia, I read about Mean ...
Oskar Weber's user avatar
3 votes
1 answer
145 views

Is it ok to select models using MASE and present the metric to client area as MAPE?

My question follow this one. Metrics such as MASE and MSE have better properties than MAPE. ...
Guilherme Parreira's user avatar
2 votes
1 answer
750 views

Understanding MASE and sktime implementation

Sorry, but after reading the Hyndman's paper here I don't understand a couple of key points. The term "in-sample" The implementation in Python using sktime Regarding the first point, I ...
isg75's user avatar
  • 31
2 votes
1 answer
192 views

Why is my MASE systematically equal to 1?

Hello I am trying to calculate the MASE of a store at hourly level. My questions are below: If I sum up the different values, it sums to 24 (number of hours) and the average comes to 1. What am I ...
Saba Muhammad Ali's user avatar
1 vote
0 answers
116 views

How to calculate MASE based on tsCV output in R?

I have a large number of time series with different properties, and applied tsCV() function to them based on different models. Now I need to compare the forecast accuracy based on tsCV output. But I ...
Mattymat's user avatar
1 vote
1 answer
135 views

Why can't I reproduce the MASE of a time series manually as in done in the Fable package?

I am trying to produce the Mean Absolute Scaled Error (MASE) for a custom time series model to compare its performance in forecasting several indicators with different units. I wanted to check my ...
Alex's user avatar
  • 11
4 votes
1 answer
184 views

Model performance in time-series forecasting with some outliers

I'm creating forecasts for products where some of them have large seasonal spike during times like Christmas and/or Easter but relatively low sales volume on other times. For this particular product ...
Viðar Ingason's user avatar
1 vote
1 answer
472 views

Did I calculate the non-seasonal Mean Absolute Scaled Error (MASE) correctly?

This is the formula: Here is the link to page in the book. I am not confident I am interpreting the formula correctly. Below is my data: I calculate the non-seasonal MASE to be 1.125. Is this ...
M_Neelakandan's user avatar
2 votes
1 answer
536 views

Interpretation of scaled error measures

can someone give me an explanation on how one would interpret the result of a scaled error measure. For example the Mean Absolute Scaled Measure (MASE). The numerator is the mean absolute error and ...
ktl12's user avatar
  • 41
2 votes
1 answer
342 views

Out of sample MASE

When calculating the MASE, the original paper suggests using the in-sample naive forecast error for scaling of the out of sample forecast error. When i use the the MAE generated by a naive forecast on ...
Madzor's user avatar
  • 23
1 vote
2 answers
3k views

Is there any standard / criteria of good forecast measured by SMAPE and MASE?

I have built a forecasting model for a company. Since it is dedicated to practical usage, I prefer to use the relative error parameter (like MAPE, SMAPE, & MASE) as a measurement for my model ...
Defa Ihsan's user avatar
1 vote
2 answers
208 views

Computing aggregated MASE for multiple time series

I think I understand how MASE works when I have a single time series. But what if I have several, for which I want to obtain an overall accuracy measure? It's straightforward to compute an aggregate ...
Hong Ooi's user avatar
  • 8,329
0 votes
0 answers
157 views

R - How to create a forecast object from a pre-defined forecast, in order to apply accuracy() for MASE result

I am working with a forecast which has been created by Prophet. I would like to apply accuracy() to return the MASE after identifying this as the best accuracy measure for multiple forecasts, and will ...
user276999's user avatar
0 votes
0 answers
93 views

Do you clean the data before calculating MASE (Mean Absolute Scaled Error)

The denominator in the MASE calculation for seasonal data is the MAE of the seasonal naive forecast calculated in-sample. Is it common to do imputation before calculating the seasonal naive MAE or ...
Alex's user avatar
  • 2,021
11 votes
1 answer
8k views

How do I decide when to use MAPE, SMAPE and MASE for time series analysis on stock forecasting

My task is to forecast future 1 month stock required for retail store, at a daily basis. How do I decide whether MAPE, SMAPE and MASE is a good metrics for the scenario? In my context, over-forecast ...
william007's user avatar
  • 1,087
1 vote
2 answers
584 views

Understanding MASE Value

I've looked through many of the other posts concerning the Mean Absolute Scaled Error (MASE) forecast metric and haven't been able to sort out my problem just yet. I'm working with some weather ...
Foxhound013's user avatar
0 votes
1 answer
993 views

Mean Absolute Scaled Error implementation on multistep time series forecast

The formula for MASE can be found here: https://en.wikipedia.org/wiki/Mean_absolute_scaled_error I am building a multi-step time series forecaster and I want to use MASE as a measure of prediction ...
Mike's user avatar
  • 227
2 votes
1 answer
5k views

How can MASE (Mean Absolute Scaled Error) score value be interpreted for non time series data?

If I have used MASE to calculate non time-series data error (as described by Dr. Rob Hyndman here), how can I know if the score received is good or not? Since it is not a time-series, a random walk ...
YonGU's user avatar
  • 21
1 vote
1 answer
750 views

MASE and handling nan-values

I'd like to ask advice on how to correctly compute Mean Absolute Scaled Error (2006, Hyndman, Rob J., and Anne B. Koehler.) over the following example: ...
Bear's user avatar
  • 135
0 votes
1 answer
46 views

Can I roll-up forecasts to analyze against benchmark methods?

There's a software product (call it Main) we use to allocate parts to various locations. The way it works is it sums up the demand history of all locations to the part level. For example, if part A ...
Angus's user avatar
  • 239
0 votes
1 answer
31 views

Is it statistically valid to compare error measures for different sized samples?

I have forecasts for different sized samples using a variety of methods like DES (Double Exponential Smoothing), SES, MA and WA (Weighted Average) to test the strength of the forecasting models. The ...
Angus's user avatar
  • 239
-1 votes
1 answer
1k views

why is the MSE error higher than MASE and MAPE?

I have a product price time series when I apply two models on them, I calculate all of MSE (Mean Squared Error), MASE (Mean Absolute Scaled Error), and MAPE (Mean Absolute Percentage Error). I noticed ...
Hana90's user avatar
  • 99
0 votes
1 answer
1k views

Confused about h-step ahead forecasts

I have 12 months of in-sample data and 12 months of out-of-sample data. I'm trying to calculate the scaled error for an h-step ahead forecast where h=1, 6 and 12. Do I just calculate the error at ...
Angus's user avatar
  • 239
0 votes
0 answers
51 views

Can MASE be calculated for a single horizon but for multiple series?

I have 600 series and I want to calculate the MASE for horizons 1, 3, 6, 8, 10 and 12. I've seen the work of Nikolaos Kourentzes where he calculates an ASE (no Mean) for each of three time horizons ...
Angus's user avatar
  • 239
0 votes
1 answer
47 views

Why am I getting different responses from MASE calculations and normality of residuals?

My company uses an inventory model (call it IM) to order parts. I have two years of data (Jan 2016 to Dec 2017) that show how many parts were actually ordered and as well as a forecast from Jan-Dec ...
Angus's user avatar
  • 239
0 votes
0 answers
84 views

Is a MASE calculation meaningful if the Test Set is all zeros?

I'm dealing with a dataset of intermittent demand (orders) for the inventory of over 6000 parts. I've extracted one of the parts just to show the type of output I'm getting for a MASE calculation. ...
Angus's user avatar
  • 239
1 vote
1 answer
219 views

Can MASE be used with a rolling forecast?

I have four years of monthly data, but I want to use three years to test the forecast of a model which uses only one year of historical data in a rolling way. For example: historical data from Jan-...
Angus's user avatar
  • 239
0 votes
1 answer
986 views

Is MASE specified only to time series data

Will it be correct to use Mean absolute scaled error in non time series data? I've got a set which contains a lot of zeros, so errors like MAPE can not be used here. MASE based on difference between ...
Roberto's user avatar
  • 113
1 vote
1 answer
890 views

Multi-input, multi-output time series regression loss using MASE

I have a time series regression that, given a set of lagged values, predicts all of the values from one step ahead to a given horizon. I'm trying to calculate the MASE for this, but am having a ...
Nate Diamond's user avatar
2 votes
0 answers
396 views

Why is MASE scaled by the mean absolute error produced by a naive forecast calculated on the in-sample data

Wouldn't a better scaling factor be with the MAE produced by a naive forecast on the test data itself? When evaluating MASE for the training set, this essentially becomes a comparison for the ...
KRS-fun's user avatar
  • 363
3 votes
1 answer
3k views

How to interpret MASE for longer horizon forecasts?

After looking at Hyndman and Koehler, 2006 and applying the metric to my own data, I have been convinced that MASE is a better metric for evaluating forecast error than the method I had been ...
Barker's user avatar
  • 1,212
7 votes
1 answer
4k views

Can I use mean absolute scaled error (MASE) from the accuracy function for time series cross validation?

I am using the "forecast" package in R to forecast time series data. I am programming some time series cross validation based off of reading resources from Rob J Hyndman. The last paragraph on page 7 ...
DataJack's user avatar
  • 293
1 vote
0 answers
1k views

Is MASE / Mean Absolute Scaled Error and the usually described denominator appropriate in this case?

I have read through: http://robjhyndman.com/hyndsight/smape/ and https://www.otexts.org/fpp/2/5 and a lot of Hyndman, R. J. and Koehler, A. B. (2006) ‘Another look at measures of forecast accuracy’, ...
greenglass's user avatar
1 vote
1 answer
1k views

Calculate MASE for time series with multiple seasonalities

What is an appropriate way to calculate the MASE accuracy measure (Link) for a time series with multiple seasonalities? For example: daily data with a strong weekly pattern and annual pattern. ...
RandomDude's user avatar
1 vote
0 answers
169 views

Time series data pre processing - model improves?

I have a time series that includes some rare extreme values. We are talking about daily data, in total 1461 observations and 11 extreme values. I adjusted those 11 values with a multiple regression. ...
RandomDude's user avatar
0 votes
1 answer
5k views

Acceptable limit for MASE

What are good sign of fit from result of forecast::accuracy. How to interpret ...
Shiv's user avatar
  • 141
1 vote
0 answers
467 views

Calculating MASE Scores for Individual Participants

I have run a controlled experiment in which 30 participants made one step ahead forecasts under two separate conditions. I want to know if there is a difference in error scores between the two ...
user2294337's user avatar
4 votes
1 answer
20k views

ARIMA: How to interpret MAPE?

I am using the forecast package in R to generate an ARIMA model for my data. I started with the auto.arima function for a try and got a ARIMA(1,1,2) model. ...
MikeHuber's user avatar
  • 1,239
0 votes
0 answers
153 views

Mean Absolute Scaled Error [duplicate]

Right now, I am analyzing the prediction quality of a dynamic model that has variables with different units (e.g. $x_{1,t}$ is in meters, $x_{2,t}$ is in kilograms etc.). I have discovered a great ...
Karel Macek's user avatar
  • 2,856
35 votes
2 answers
46k views

Interpretation of mean absolute scaled error (MASE)

Mean absolute scaled error (MASE) is a measure of forecast accuracy proposed by Koehler & Hyndman (2006). $$MASE=\frac{MAE}{MAE_{in-sample, \, naive}}$$ where $MAE$ is the mean absolute error ...
Richard Hardy's user avatar
9 votes
1 answer
13k views

Time series forecasting accuracy measures: MAPE and MASE

We come to this toy example showing MAPE and MASE are not consistent when measuring forecasting accuracy. Data consist of 100 white noise and 100 $AR(1)$ time series with length $N=500$, mean $\mu=1$ ...
yanfei kang's user avatar
1 vote
1 answer
1k views

Alternative to MAPE when the data is not a time series

I have a data set where many of the actual values are zero, so I can't use MAPE. It's not a time series, so I can't use MASE ala our very own Rob Hyndman. Is there another alternative to MAPE that I ...
JenSCDC's user avatar
  • 1,075
1 vote
1 answer
455 views

R : accuracy.gts, no MASE with monthly data

I have a problem similar to the one presented in this post : https://stackoverflow.com/questions/11092536/forecast-accuracy-no-mase-with-two-vectors-as-arguments even if it's maybe not related. I'm ...
Alex's user avatar
  • 11