Questions tagged [mae]
The Mean Absolute Error (MAE) is a point forecast accuracy measure. In the forecasting literature, Mean Absolute Deviation (MAD) is used interchangeably.
80
questions
1
vote
1
answer
20
views
How to approximate MAE of monthly values from MAE of daily values?
Suppose I have the Mean Absolute Error (MAE) of daily values for a period of, say, 1 year. Assume the errors are normally distributed.
The value for a month is equal to the sum of the values for each ...
1
vote
0
answers
19
views
MAPE does not take into account the range of the output?
I have a time-series regression model where the output is always in the range of 6000-6050. After training my model, I get a Mean Absolute Error of around 18 and hence, very low Mean Absolute ...
- 11
1
vote
0
answers
45
views
is Mean Absolute Error (MAE) unique?
Reading this answer as to why minimizing MAE results in median forecasts, I did not fully understand why MAE is not unique!
What is exactly meant by this?
Can someone give a specific example of ...
- 11
1
vote
1
answer
23
views
Showing RMSE and MAE results as percentage error
I have the results of RMSE and MAE from different spatial interpolation methods as a monthly averages (See the figure below). As ...
- 11
3
votes
2
answers
142
views
Alternative to mean absolute percentage error (MAPE)
MAPE metric has problems when the actual value to be predicted is very small. In the extreme when the actual value is 0 then MAPE will be infinity (if the prediction is not exactly 0). What about this ...
- 883
1
vote
0
answers
31
views
Is there a single regression quality metric for the median and the 95% percentile?
I want to evaluate the quality of prediction of two values the median and 95% percentile of a distribution. Is there a standard way to do this? I have thought about using "Mean Mean Average Error&...
- 133
0
votes
0
answers
21
views
Estimating empirically the Wasserstein distance
I have a dataset of the form $\{x_i,y_i,y_i'\}$, where $y_i\sim p(\cdot|x_i)$ and $y_i'\sim q(\cdot|x_i)$, while $x_i$ itself has a distribution $d(\cdot)$
Is there I way to estimate
$$\mathbb E_{x\...
- 101
0
votes
1
answer
16
views
Why can the aggregated prediction of two bad models be better than the one two very good models?
I am modeling two separated processes (e.g., number of events A and B). As jointly modeling both processes is not possible, I estimate for each process three different models.
When I look at the three ...
- 882
0
votes
1
answer
63
views
What optimisation method is used in ordered logistic regression?
I am using the polr function in R to create an ordered logistic model and am curious to know what its optimisation method is?
It seems to perform better than other models I have tested it against when ...
- 1
0
votes
0
answers
13
views
Measuring errors of relative frequency distribution over a time series
I have 2 time series, in each I have the number of visits in a certain place - one is the real number, one is a processed sample. Since the real numbers are in xxK visits per week and the sample is in ...
- 75
1
vote
1
answer
50
views
Interpretation of mean absolute error in rpart
I ran rpart on my dataset (3000x9) to predict adolescent GPA (continuous), made predictions on test data, and found the mean absolute error to be in the range of 0.45-0.48. Does this mean that the ...
- 119
1
vote
0
answers
57
views
High RMSE and High MAE in Autoencoder Regression
I have been developing a simple autoencoder model using PyTorch by which I am training the reconstructed output to be the same and input and also do regression on the hidden layer to predict a single ...
6
votes
2
answers
2k
views
Why getting very high values for MSE/MAE/MAPE when R2 score is very good
I am applying different regression models (RF, Knn, etc) on some well-known datasets (bike sharing, diabetics, etc). The value of R2 is very good. From the R2 score,...
- 203
4
votes
1
answer
91
views
Model performance in time-series forecasting with some outliers
I'm creating forecasts for products where some of them have large seasonal spike during times like Christmas and/or Easter but relatively low sales volume on other times. For this particular product ...
- 407
0
votes
1
answer
149
views
Using standardized values (z-score) for MAE (Mean absolute error)
I have two models/indices that try to predict observed values. I've compared them using correlation and regression, but I'd like to use MAE (Mean absolute error) to asses which of them is more closer ...
- 1
0
votes
0
answers
37
views
Out-of-sample MSE and MAE for volatility forecasting [duplicate]
I have been searching through the whole CrossValidated but couldn't find the answer.
I want to test out-of-sample the volatility forecasts (if it means something ARCH-like ones, MSGARCH, Multifractal ...
- 21
3
votes
0
answers
490
views
Why is there no improvement when training Xgboost with pseudo-Huber loss?
In this StackOverflow post I asked if there was something wrong with my syntax when training an XGboost model (in R) with the native pseudo-Huber loss ...
- 883
0
votes
1
answer
214
views
Does Data Normalization affect RMSE & MAE?
Suppose I am using two different methods to perform regression, such as ARIMA (1) and a Neural Network (2), and I am using the RMSE and MAE metrics to measure accuracy. For method (1), I do not ...
0
votes
1
answer
116
views
Choosing metric for regression problem RMSE or MAE
I have a regression task where I want to predict no of product sold by manufacturer. My data is time series with each day having a certain no of units sold.
I'm slightly torn as to which metric to use....
- 73
0
votes
1
answer
198
views
Is this the right way to evaluate MSE, RMSE, MAE, MAPE after creating a model on data passed through a standard scaler?
Create the Scaler object
scaler = preprocessing.StandardScaler()
Fit your training data on the scaler object
...
0
votes
1
answer
1k
views
What is the interpretation of the negative value for the Normalized Mean Absolute Error (nMAE) metric?
I am using the normalized mean absolute error metric for evaluating my results. The data I use is in time-series form. Their trend may be increasing or decreasing over time. All the values are ...
- 101
3
votes
2
answers
960
views
Can MAE be reported in percentage?
In Wang et al. "Predicting New Workload or CPU Performance by Analyzing Public Datasets" (2019), the MAE is measured in a standardized space (Mean=0, SD=1). This is discussed in 3.4 Metrics, ...
- 183
0
votes
2
answers
783
views
MAPE is better but MAE is worse in regression models
I am working on a regression problem to predict price of the vehicle based on its features. I have been experimenting with several trials but in one of them, MAPE (Mean Absolute Percentage Error) is ...
- 103
1
vote
1
answer
351
views
What is mean average error?
I saw that in some articles but I couldn't find any definition. Its unit is in the following form:
MAE/mean(MAE)
- 111
1
vote
0
answers
84
views
huge difference between RMSE and MAE in non-linear regressors
I am building non-linear regression models such as a Random Forest regressor , a KNN regressor or a SVM using a RBF kernel and I decided to use both RMSE and MAE as evaluation metrics.
I know that a ...
0
votes
0
answers
391
views
Mean average error normalized by the standard deviation of the target
I'm working on a regression model, aiming at prediction age from structured data.
I'm using the mean average error (MAE) as evaluation metric, and want to compare my performances with state-of-the-art ...
2
votes
1
answer
207
views
MAE regression gives biased regression parameters for symmetric error?
Consider a linear model,
$$
y_i = \beta_0 + \beta_1x_{1i} + \beta_2x_{2i} + \epsilon_i.
$$
From the Gauss-Markov theorem, I know that, under nice conditions, the $\hat{\beta}_{OLS}=(X^TX)^{-1}X^Ty$ ...
- 31.3k
1
vote
0
answers
40
views
How should i compare this MAE results
I'm in need of help to correctly interpret these results.
I am analyzing a time series of crime data. The series has 351,980 events.
These are the results of the predictions using a regression model:...
- 11
1
vote
1
answer
14k
views
How do I interpret mean absolute error (MAE) or mean absolute percentage error (MAPE) in layman words?
For example, I am predicting a score that can have value from 0 to 100. Lets assume MAPE = 10...
- 907
2
votes
1
answer
222
views
Scaling the MAE by the mean of non zero points for intermittent data
I am currently trying to find a way of scaling the MAE for my intermittent data.
The data is always greater than 0 and is intermittent, with long periods of zeros.
I have read a few papers that ...
- 21
2
votes
1
answer
100
views
What does it mean when MAE improved while RMSE worsened?
I am comparing two models:
One is a black box that I cannot understand, the other is a GLM.
How can I describe why the differences are like this? Is the GLM performing worse than the blackbox?
- 135
1
vote
1
answer
899
views
Quantile loss 50th is MAE, is it? [duplicate]
I'm not sure the above sentence is true, but I read it here, here and here that quantile loss function percentile 0.5 is MAE(mean absolute error), Is it true(Yes or No)? and How?
- 111
2
votes
2
answers
373
views
Comparison of MAE and Mean to illustrate the error magnitude
I have predicted a time series with positive, zero and negative values. As error measurement I used the Mean Absolute Error (MAE).
In order to give the reader of my paper a better understanding ...
2
votes
2
answers
51
views
Learning Curves using different models
I am running repeated K-fold Cross-Validation on my dataset using different models. My problem is a regression problem and I am counting on the error metric MAE. I do know that some models may behave ...
- 501
20
votes
2
answers
3k
views
Can someone give the intuition behind Mean Absolute Error and the Median? [duplicate]
I do not understand the intuition behind why the median is the best estimate if we are going to judge prediction accuracy using the Mean Absolute Error. Let's say you have a random variable $X$ and ...
- 2,513
4
votes
1
answer
733
views
Dividing the MAE by the average of the values
I would like to parse the MAE (Mean Absolute Error) to a percentage value. I know there is the MAPE (Mean Absolute Percentage Error), however it has some drawbacks as going to infinity if one of my ...
10
votes
2
answers
3k
views
Can RMSE and MAE have the same value?
I am implementing cross validation and calculating error metrics such as RMSE, $R^2$, MAE, MSE, etc.
Can RMSE and MAE have the same value?
- 501
0
votes
1
answer
578
views
RSME, MAE and prediction interval [closed]
Could someone please clarify, whether it is appropriate to define a prediction interval or an equivalent for an RMSE and MAE measure. If so, could you please suggest how such an interval is defined.
- 1
2
votes
2
answers
623
views
MSE Intuition and Interpretation
I've got a very small question. Say I'm making a linear regression model. When I test the model with a testing set, I get an MSE of 4.31 (arbitrary). What do I interpret from this? As in, what does ...
- 25
1
vote
0
answers
86
views
Error distribution for Huber Regression
For linear regression there's an assumption that error terms come from normal distribution. so that $Y = aX + b + \epsilon$, where $\epsilon$ has normal distribution with mean zero and certain ...
- 11
1
vote
1
answer
132
views
a measure for MAE (of a regression)
I'm running a grid search, in order to fine-tune a NN hyper parameters.
the question is:
the MAE values I get from the trainings are too close.
since I have the statistical attributes of the target ...
- 11
0
votes
3
answers
2k
views
Standardised mean absolute error (SMAE) and how to calculate it?
I am using the mean absolute error
mean(abs(obs - pred))
as one of the measures assessing the fit of my model. I would also like to have a standardised measure ...
- 151
5
votes
2
answers
429
views
What is the best point forecast for gamma distributed data?
I believe that the values I am forecasting are gamma distributed with shape $k>0$ and scale $\theta>0$. I need a point forecast (i.e., a one-number summary) that minimizes the expected error. ...
- 96.2k
6
votes
2
answers
731
views
What is the best point forecast for lognormally distributed data?
I believe that the values I am forecasting are lognormally distributed with log-mean $\mu$ and log-variance $\sigma^2$. I need a point forecast (i.e., a one-number summary) that minimizes the expected ...
- 96.2k
2
votes
1
answer
5k
views
calculating overall error in k-fold cross validation
when using k-fold cross validation i thought the overall error was equal to the mean of errors of each fold. the error being anything from MAE and RMSE to NDCG,F-measure, precision and recall. however ...
- 23
0
votes
2
answers
2k
views
Mean Absolute Error in Random Forest Regression
I am new to the whole ML scene and am trying to resolve the Allstate Kaggle challenge to get a better feeling for the Random Forest Regression technique.
The challenge is evaluated based on the MAE ...
- 103
0
votes
1
answer
64
views
What statistical test do I need for comparing forecasted data with actual data?
I’m currently completing my dissertation and need to compare forecasted wave height to the actual wave height.
However I am unsure what statistical test to use.
Thanks,
Jess
2
votes
2
answers
4k
views
Does normalisation affect the values of Mean Squared Error, Mean Absolute Percentage Error etc.?
I have a large amount of data, divided into folders and files. Each file has 56 features/columns and around 10,000 rows. The data is normalised (values between -1 and 1) and some of the features have ...
- 123
38
votes
3
answers
10k
views
Why does minimizing the MAE lead to forecasting the median and not the mean?
From the Forecasting: Principles and Practice textbook by Rob J Hyndman and George Athanasopoulos, specifically the section on accuracy measurement:
A forecast method that minimizes the MAE will ...
- 1,232
1
vote
1
answer
367
views
MSE or MAE absolute and relative performance
I'm comparing different ARMA-GARCH specification out-of-sample in order to understand whether the more "parsimonious" models prescribed by BIC do not perform more poorly than the more "expensive" ones ...
- 71