Questions tagged [mase]
The Mean Absolute Scaled Error (MASE) was proposed by Hyndman & Koehler (2006, *International Journal of Forecasting*) as a scale free accuracy measure for point forecasts. It is defined as the ratio of the MAE to the one-step MAE achieved *in-sample* (this is frequently gotten wrong) by a simple benchmark method (often the naive random walk forecast).
42
questions
3
votes
1
answer
93
views
Is it ok to select models using MASE and present the metric to client area as MAPE?
My question follow this one.
Metrics such as MASE and MSE have better properties than MAPE.
...
2
votes
1
answer
595
views
Understanding MASE and sktime implementation
Sorry, but after reading the Hyndman's paper here I don't understand a couple of key points.
The term "in-sample"
The implementation in Python using sktime
Regarding the first point, I ...
- 31
2
votes
1
answer
170
views
Why is my MASE systematically equal to 1?
Hello I am trying to calculate the MASE of a store at hourly level. My questions are below:
If I sum up the different values, it sums to 24 (number of hours) and the average comes to 1. What am I ...
1
vote
0
answers
111
views
How to calculate MASE based on tsCV output in R?
I have a large number of time series with different properties, and applied tsCV() function to them based on different models. Now I need to compare the forecast accuracy based on tsCV output. But I ...
- 11
0
votes
1
answer
119
views
Why can't I reproduce the MASE of a time series manually as in done in the Fable package?
I am trying to produce the Mean Absolute Scaled Error (MASE) for a custom time series model to compare its performance in forecasting several indicators with different units. I wanted to check my ...
- 1
4
votes
1
answer
177
views
Model performance in time-series forecasting with some outliers
I'm creating forecasts for products where some of them have large seasonal spike during times like Christmas and/or Easter but relatively low sales volume on other times. For this particular product ...
- 437
0
votes
1
answer
392
views
Did I calculate the non-seasonal Mean Absolute Scaled Error (MASE) correctly?
This is the formula:
Here is the link to page in the book. I am not confident I am interpreting the formula correctly. Below is my data:
I calculate the non-seasonal MASE to be 1.125. Is this ...
2
votes
1
answer
437
views
Interpretation of scaled error measures
can someone give me an explanation on how one would interpret the result of a scaled error measure.
For example the Mean Absolute Scaled Measure (MASE). The numerator is the mean absolute error and ...
- 41
2
votes
1
answer
262
views
Out of sample MASE
When calculating the MASE, the original paper suggests using the in-sample naive forecast error for scaling of the out of sample forecast error.
When i use the the MAE generated by a naive forecast on ...
- 23
1
vote
2
answers
2k
views
Is there any standard / criteria of good forecast measured by SMAPE and MASE?
I have built a forecasting model for a company. Since it is dedicated to practical usage, I prefer to use the relative error parameter (like MAPE, SMAPE, & MASE) as a measurement for my model ...
- 19
0
votes
2
answers
180
views
Computing aggregated MASE for multiple time series
I think I understand how MASE works when I have a single time series. But what if I have several, for which I want to obtain an overall accuracy measure?
It's straightforward to compute an aggregate ...
- 7,979
0
votes
0
answers
152
views
R - How to create a forecast object from a pre-defined forecast, in order to apply accuracy() for MASE result
I am working with a forecast which has been created by Prophet. I would like to apply accuracy() to return the MASE after identifying this as the best accuracy measure for multiple forecasts, and will ...
0
votes
0
answers
92
views
Do you clean the data before calculating MASE (Mean Absolute Scaled Error)
The denominator in the MASE calculation for seasonal data is the MAE of the seasonal naive forecast calculated in-sample.
Is it common to do imputation before calculating the seasonal naive MAE or ...
- 1,921
11
votes
1
answer
8k
views
How do I decide when to use MAPE, SMAPE and MASE for time series analysis on stock forecasting
My task is to forecast future 1 month stock required for retail store, at a daily basis. How do I decide whether
MAPE, SMAPE and MASE
is a good metrics for the scenario?
In my context, over-forecast ...
- 1,067
1
vote
2
answers
546
views
Understanding MASE Value
I've looked through many of the other posts concerning the Mean Absolute Scaled Error (MASE) forecast metric and haven't been able to sort out my problem just yet.
I'm working with some weather ...
- 111
0
votes
1
answer
943
views
Mean Absolute Scaled Error implementation on multistep time series forecast
The formula for MASE can be found here: https://en.wikipedia.org/wiki/Mean_absolute_scaled_error
I am building a multi-step time series forecaster and I want to use MASE as a measure of prediction ...
- 217
2
votes
1
answer
4k
views
How can MASE (Mean Absolute Scaled Error) score value be interpreted for non time series data?
If I have used MASE to calculate non time-series data error (as described by Dr. Rob Hyndman here), how can I know if the score received is good or not? Since it is not a time-series, a random walk ...
- 21
1
vote
1
answer
686
views
MASE and handling nan-values
I'd like to ask advice on how to correctly compute Mean Absolute Scaled Error (2006, Hyndman, Rob J., and Anne B. Koehler.) over the following example:
...
- 135
0
votes
1
answer
44
views
Can I roll-up forecasts to analyze against benchmark methods?
There's a software product (call it Main) we use to allocate parts to various locations. The way it works is it sums up the demand history of all locations to the part level. For example, if part A ...
- 239
0
votes
1
answer
30
views
Is it statistically valid to compare error measures for different sized samples?
I have forecasts for different sized samples using a variety of methods like DES (Double Exponential Smoothing), SES, MA and WA (Weighted Average) to test the strength of the forecasting models. The ...
- 239
-1
votes
1
answer
1k
views
why is the MSE error higher than MASE and MAPE?
I have a product price time series when I apply two models on them, I calculate all of MSE (Mean Squared Error), MASE (Mean Absolute Scaled Error), and MAPE (Mean Absolute Percentage Error). I noticed ...
- 99
0
votes
1
answer
1k
views
Confused about h-step ahead forecasts
I have 12 months of in-sample data and 12 months of out-of-sample data. I'm trying to calculate the scaled error for an h-step ahead forecast where h=1, 6 and 12. Do I just calculate the error at ...
- 239
0
votes
0
answers
50
views
Can MASE be calculated for a single horizon but for multiple series?
I have 600 series and I want to calculate the MASE for horizons 1, 3, 6, 8, 10 and 12. I've seen the work of Nikolaos Kourentzes where he calculates an ASE (no Mean) for each of three time horizons ...
- 239
0
votes
1
answer
47
views
Why am I getting different responses from MASE calculations and normality of residuals?
My company uses an inventory model (call it IM) to order parts. I have two years of data (Jan 2016 to Dec 2017) that show how many parts were actually ordered and as well as a forecast from Jan-Dec ...
- 239
0
votes
0
answers
81
views
Is a MASE calculation meaningful if the Test Set is all zeros?
I'm dealing with a dataset of intermittent demand (orders) for the inventory of over 6000 parts. I've extracted one of the parts just to show the type of output I'm getting for a MASE calculation. ...
- 239
1
vote
1
answer
216
views
Can MASE be used with a rolling forecast?
I have four years of monthly data, but I want to use three years to test the forecast of a model which uses only one year of historical data in a rolling way. For example:
historical data from Jan-...
- 239
0
votes
1
answer
966
views
Is MASE specified only to time series data
Will it be correct to use Mean absolute scaled error in non time series data? I've got a set which contains a lot of zeros, so errors like MAPE can not be used here. MASE based on difference between ...
- 113
0
votes
1
answer
859
views
Multi-input, multi-output time series regression loss using MASE
I have a time series regression that, given a set of lagged values, predicts all of the values from one step ahead to a given horizon. I'm trying to calculate the MASE for this, but am having a ...
- 145
2
votes
0
answers
373
views
Why is MASE scaled by the mean absolute error produced by a naive forecast calculated on the in-sample data
Wouldn't a better scaling factor be with the MAE produced by a naive forecast on the test data itself?
When evaluating MASE for the training set, this essentially becomes a comparison for the ...
- 363
2
votes
1
answer
3k
views
How to interpret MASE for longer horizon forecasts?
After looking at Hyndman and Koehler, 2006 and applying the metric to my own data, I have been convinced that MASE is a better metric for evaluating forecast error than the method I had been ...
- 1,162
7
votes
1
answer
4k
views
Can I use mean absolute scaled error (MASE) from the accuracy function for time series cross validation?
I am using the "forecast" package in R to forecast time series data. I am programming some time series cross validation based off of reading resources from Rob J Hyndman.
The last paragraph on page 7 ...
- 293
1
vote
0
answers
1k
views
Is MASE / Mean Absolute Scaled Error and the usually described denominator appropriate in this case?
I have read through:
http://robjhyndman.com/hyndsight/smape/ and https://www.otexts.org/fpp/2/5
and a lot of
Hyndman, R. J. and Koehler, A. B. (2006) ‘Another look at measures of forecast accuracy’, ...
- 225
1
vote
1
answer
1k
views
Calculate MASE for time series with multiple seasonalities
What is an appropriate way to calculate the MASE accuracy measure (Link) for a time series with multiple seasonalities?
For example: daily data with a strong weekly pattern and annual pattern.
...
- 505
1
vote
0
answers
168
views
Time series data pre processing - model improves?
I have a time series that includes some rare extreme values. We are talking about daily data, in total 1461 observations and 11 extreme values. I adjusted those 11 values with a multiple regression. ...
- 505
0
votes
1
answer
5k
views
Acceptable limit for MASE
What are good sign of fit from result of forecast::accuracy.
How to interpret
...
- 141
1
vote
0
answers
467
views
Calculating MASE Scores for Individual Participants
I have run a controlled experiment in which 30 participants made one step ahead forecasts under two separate conditions. I want to know if there is a difference in error scores between the two ...
- 11
4
votes
1
answer
20k
views
ARIMA: How to interpret MAPE?
I am using the forecast package in R to generate an ARIMA model for my data. I started with the auto.arima function for a try and got a ARIMA(1,1,2) model.
...
- 1,229
0
votes
0
answers
152
views
Mean Absolute Scaled Error [duplicate]
Right now, I am analyzing the prediction quality of a dynamic model that has variables with different units (e.g. $x_{1,t}$ is in meters, $x_{2,t}$ is in kilograms etc.). I have discovered a great ...
- 2,738
34
votes
2
answers
44k
views
Interpretation of mean absolute scaled error (MASE)
Mean absolute scaled error (MASE) is a measure of forecast accuracy proposed by Koehler & Hyndman (2006).
$$MASE=\frac{MAE}{MAE_{in-sample, \, naive}}$$
where $MAE$ is the mean absolute error ...
- 64.6k
9
votes
1
answer
13k
views
Time series forecasting accuracy measures: MAPE and MASE
We come to this toy example showing MAPE and MASE are not consistent when measuring forecasting accuracy.
Data consist of 100 white noise and 100 $AR(1)$ time series with length $N=500$, mean $\mu=1$ ...
- 427
1
vote
1
answer
1k
views
Alternative to MAPE when the data is not a time series
I have a data set where many of the actual values are zero, so I can't use MAPE. It's not a time series, so I can't use MASE ala our very own Rob Hyndman.
Is there another alternative to MAPE that I ...
- 1,067
1
vote
1
answer
453
views
R : accuracy.gts, no MASE with monthly data
I have a problem similar to the one presented in this post : https://stackoverflow.com/questions/11092536/forecast-accuracy-no-mase-with-two-vectors-as-arguments
even if it's maybe not related.
I'm ...
- 11