2k views

Can someone give the intuition behind Mean Absolute Error and the Median? [duplicate]

I do not understand the intuition behind why the median is the best estimate if we are going to judge prediction accuracy using the Mean Absolute Error. Let's say you have a random variable $X$ and ...
422 views

Quantile loss 50th is MAE, is it? [duplicate]

I'm not sure the above sentence is true, but I read it here, here and here that quantile loss function percentile 0.5 is MAE(mean absolute error), Is it true(Yes or No)? and How?
42k views

What are the shortcomings of the Mean Absolute Percentage Error (MAPE)?

The Mean Absolute Percentage Error (mape) is a common accuracy or error measure for time series or other predictions, $$\text{MAPE} = \frac{100}{n}\sum_{t=1}^n\frac{|A_t-F_t|}{A_t}\%,$$ where $A_t$ ...
2k views

(Why) Is absolute loss not a proper scoring rule?

Brier score is a proper scoring rule and is, at least in the binary classification case, square loss. $$Brier(y,\hat{y}) = \frac{1}{N} \sum_{i=1}^N\big\vert y_i -\hat{y}_i\big\vert^2$$ Apparently this ...
1k views

How to optimize MAPE in regression algorithms

I have a regression task where the label is varying from about 0.001 to 1000. One of the feature called group, for example, group A corresponding label from 0-0.1 and group G corresponding label from ...
437 views

Need advice on evaluating forecast accuracy in R

I'm trying to evaluate some software for forecast accuracy. It works by summing up all the orders from a number of locations for each month, then determines the best model out of a series of models ...
2k views

time series forecasting in R for a period less than 2 years(18 months) which is totally random

I'm working on a project of forecasting. I have the count of the purchase order for an 18 months period of time. I'm attempting to create a forecast from time series data that has observations only on ...
1k views

Should I convert the number after log transformation back to the original for calculating RMSE?

When I built the model, I applied the log transformation to all variables including the dependent variables. Now, I'm calculating the RMSE for the evaluation, and the result is in the log format, ...
931 views

Higher RMSE lower MAPE

I have a time series model that forecast next K days. For example when I forecast next 50 days my MAPE is 20.3% and RMSE is 2943 and when I forecast next 200 days is the MAPE is 10.25 % but RMSE is ...
317 views

Predictive distribution: What can we say about the prediction?

I am wondering what could we obtain more from the predictive distribution. Give a set of data, say $\mathcal{D}=\{(x_i,y_i)\}$, we want to predict the value s$Y_{new}$ at new locations, say $X_{new}$. ...
520 views

MAPE and SMAPE shift invariance (bias)

MAPE (Mean Absolute Percentage Error) and SMAPE (Symmetric Mean Absolute Percentage Error) both are sensitive when the TRUE ...
258 views

What measurement of error should I use to compare predicted model results to actual measurements (and why)?

Lets say I have a time-series dataset of measurements g that varies with time t and I also have a time-series dataset of predictions of these measurements (lets call this g1) that again varies with ...
256 views

Expected Predicted Error (EPE) with L1 loss

In Element of Statistical learning it is saying on page 20, equation 2.18. That using the L1 norm instead of the usual L2 norm leads to an $f(X)$ optimising the EPE being the median instead of the ...