Questions tagged [mae]
The Mean Absolute Error (MAE) is a point forecast accuracy measure. In the forecasting literature, Mean Absolute Deviation (MAD) is used interchangeably.
0
votes
1answer
66 views
a measure for MAE (of a regression)
I'm running a grid search, in order to fine-tune a NN hyper parameters.
the question is:
the MAE values I get from the trainings are too close.
since I have the statistical attributes of the target ...
0
votes
3answers
52 views
Standardised mean absolute error (SMAE) and how to calculate it?
I am using the mean absolute error
mean(abs(obs - pred))
as one of the measures assessing the fit of my model. I would also like to have a standardised measure ...
4
votes
1answer
60 views
What is the best point forecast for gamma distributed data?
I believe that the values I am forecasting are gamma distributed with shape $k>0$ and scale $\theta>0$. I need a point forecast (i.e., a one-number summary) that minimizes the expected error. ...
3
votes
1answer
49 views
What is the best point forecast for lognormally distributed data?
I believe that the values I am forecasting are lognormally distributed with log-mean $\mu$ and log-variance $\sigma^2$. I need a point forecast (i.e., a one-number summary) that minimizes the expected ...
1
vote
1answer
169 views
calculating overall error in k-fold cross validation
when using k-fold cross validation i thought the overall error was equal to the mean of errors of each fold. the error being anything from MAE and RMSE to NDCG,F-measure, precision and recall. however ...
0
votes
1answer
212 views
Mean Absolute Error in Random Forest Regression
I am new to the whole ML scene and am trying to resolve the Allstate Kaggle challenge to get a better feeling for the Random Forest Regression technique.
The challenge is evaluated based on the MAE ...
0
votes
0answers
27 views
How to best evaluate a cross validation of a logistic regression using cbind
I ran a logistic GLMM using cbind for the response:
...
0
votes
0answers
19 views
Calculate the Confidence Interval for the Error of Model
I am not sure I am thinking about my problem the right way, so I am looking for the right approach.
I have a data set that, for the sake of argument, has a mean of 1 and a standard deviation of $\...
0
votes
1answer
28 views
What statistical test do I need for comparing forecasted data with actual data?
I’m currently completing my dissertation and need to compare forecasted wave height to the actual wave height.
However I am unsure what statistical test to use.
Thanks,
Jess
1
vote
1answer
67 views
Does normalisation affect the values of Mean Squared Error, Mean Absolute Percentage Error etc.?
I have a large amount of data, divided into folders and files. Each file has 56 features/columns and around 10,000 rows. The data is normalised (values between -1 and 1) and some of the features have ...
0
votes
0answers
25 views
Interpreting the Evaluation Model for Decision Forest Tree
I am working on a project to find Productivity of a person in near future and i am using Decision Forest Tree in Azure Machine Learning studio. I have a good amount of sample data (5459 rows) & 9 ...
0
votes
0answers
105 views
Mean absolute deviation intervals vs. standard deviation?
I have used standard deviation and RMSE to calculate a confidence interval in a time series model. I am also interested in trying out MAE as it seems to catch more anomalies but I'm not sure how I ...
13
votes
2answers
2k views
Why does minimizing the MAE lead to forecasting the median and not the mean?
From the Forecasting: Principles and Practice textbook by Rob J Hyndman and George Athanasopoulos, specifically the section on accuracy measurement:
A forecast method that minimizes the MAE will ...
1
vote
1answer
195 views
MSE or MAE absolute and relative performance
I'm comparing different ARMA-GARCH specification out-of-sample in order to understand whether the more "parsimonious" models prescribed by BIC do not perform more poorly than the more "expensive" ones ...
0
votes
1answer
456 views
MAE and Precision for Collaborative Filtering Recommender Systems
I have got a question concerning Recommender Systems and Evaluation Metrics. I tested a few collaborative filtering recommendation algorithms on dataset containing amazon ratings.
Here you can see MAE ...
1
vote
0answers
222 views
How to evaluate metrics values for regression validation (MAE, MSE and SSE)? [closed]
I am new to ML and everywhere I look for regression examples I see basically 3 different metrics to evaluate regression: Mean Absolute Error, ...
1
vote
0answers
252 views
Measuring accuracy for forecasting in R [closed]
I have a dataset of restaurant orders. In that data set I need to predict the outcome of the next 12 months, i.e. how much order will be given.
using the following test
...
1
vote
1answer
3k views
Mean Absolute Error (MAE) derivative
$MAE=|y_{pred} - y_{true}|$
$\dfrac{dMAE}{dy_{pred}} = ?$
I'm trying to understand how MAE works as a loss function in neural networks using backpropogation. I know it can be used directly in some ...
3
votes
1answer
110 views
improve performance in a regression methodology
I'm performing my own nonparametric regression in matlab and I'm wondering to switch one nuance of my methodology:
Methodology:
split Training/test set (i.e. 80/20 %)
search the best gamma ...
2
votes
0answers
82 views
Why does the MAE of my estimation improve substantially when I divide it by 10 (in a very noisy dataset)?
I have an extremely noisy dataset, and I try to minimize the MAE. Apparently, when my linear regression makes its prediction - it has something like -1000 MAE (when compared to the MAE of an all-zero ...
1
vote
1answer
1k views
Can RMSE be smaller than MAE?
Generally speaking, can RMSE be smaller than MAE?
I am calculating RMSE and MAE for my results. In two out of five methods, the RMSE is smaller than MAE. Note that I am using the same data, the same ...
1
vote
0answers
123 views
Is there an analog for the coefficient of determination with absolute error?
Typically to measure the predictive power of a model, R^2 is more useful than RMSE because a RMSE score isn't very meaningful without a basis of comparison. If you are using mean absolute error (MAE) ...
1
vote
1answer
490 views
MAE has gotten worse but RMSE is better, how should I interpret it?
I am doing some testing in recommender systems with extended epinions dataset, I implemented two models, model A give me RMSE of 0.5387 and MAE of 0.3111 and model B gave me RMSE of 0.5121 and MAE of ...
3
votes
1answer
4k views
L1 (MAE) vs L2 (MSE) when data is normalized between 0 and 1
In most of the literature, it is emphasized that the L2 norm (MSE) gives a higher error when dealing with outliers compared to the L1 norm (MAE).
But what happens when we normalize the data between 0 ...
3
votes
1answer
753 views
Can MAPE Decrease while StdDev Increases?
Can a decrease in mean absolute percentage error (MAPE) be correlated with an increase in standard deviation of the error? Is that counter-intuitive? What about MAPE vs. mean absolute error (MAE)? I'm ...
3
votes
1answer
647 views
Mean Absolute Error Cost Function in Logistic Regression
Is there any reasons not to use MAE as cost function in logistic regression?
Yes, I know about problems with optimization, I know that this estimation won't be max-likelihood estimation.
But is there ...
0
votes
1answer
171 views
Best imputation techniques (error test)
actually my study is about to choose best imputation techniques. Im comparing all the techniques using Root Mean Square Error (RMSE), Mean Aabsolute Error (MAE) and correlation (R). My problem is ...
3
votes
1answer
703 views
What is the point of Root Mean Absolute Error, RMAE, when evaluating forecasting errors?
RMAE is defined as the square root of the Mean Absolute Error (MAE). Presumably this is by analogy to Root Mean Square Error (RMSE) being defined as the square root of Mean Square Error (MSE).
But ...
2
votes
1answer
50 views
Error measure that take the time of prediction into account?
I am doing a machine learning project where I'm attempting to predict the arrival times of buses equipped with GPS at bus stops, and I am very new to statistics/data analysis. I am looking for an ...
0
votes
1answer
797 views
how to pick model, MAE or rMSE [duplicate]
Im building two models, (model1 and model2)
I trained and tested them on the same test dataset,
model1 will have mean absolute error (MAE) 10.3, rooted mean squared error (rMSE) 30.1
model2 will have ...
0
votes
1answer
266 views
Relatively low MAE, but also low R-squared? Why
I am testing forecasts against against realized values with a number of observations of 4000.
When I calculate the mean MAE, its relatively low. Around 10% deviation to the original variable.
...
1
vote
2answers
680 views
Using GARCH rolling forecast in R to calculate MAE?
My intention is to calculate the MAE for different (G)ARCH-models (comparing the one-step-ahead forecast for $\sigma$ with the absolute return that day).
The formula for MAE is actually clear, but I'...
3
votes
2answers
417 views
Large-scale MAE regression in R
I have a large, sparse dgCMatrix matrix in R:
~200,000 rows
~150,000 columns
~1,000,000,000 non-zero entries
R code to generate the matrix:
...
1
vote
2answers
223 views
How to check if specific MAE and MSE are feasible given only the real data?
I have as data some real measures, let's say: 1000, 800, 900, 1100, 900 and I have the Mean Absolute Error (MAE) and Mean Squared Error (MSE) 80 and 20000, but I don't know which are the estimated ...
0
votes
0answers
199 views
mape and mae from k-fold backtesting on time series
I performed a rolling window (i.e. do full sample, then next 4 observations until the last, and so on...) k-fold test for out of sample testing due to limited number of observations.
From the MAPE ...
5
votes
1answer
228 views
Is it possible to compute RMSE iteratively?
I am working on continuous evaluation of a regression model on streaming data from sensors. I think that Mean Absolute Error (MAE) can be found out iteratively similar to this link for averaging.
$$
...
1
vote
1answer
311 views
Why do I get identical MSE, MAE, DAC from different GARCH specification?
I estimated 5 different garch models using rugarch package as:
...
4
votes
0answers
202 views
Regression models with comparable MAE but differing R²
I have trained two regression models on the same dataset.
They perform with comparable mean absolut errors $MAE_{1,2} \approx 0.45$, but the coefficient of determination differs significantly with $R^...
8
votes
1answer
1k views
Why not using the R squared to measure forecast accuracy?
Why in literature usually the common accuracy measures like MAD, MSE, RMSE, MAPE ... are used. Why not using the $R^2$ (coefficient of determination)?
I was thinking about the difference: By using ...
0
votes
1answer
94 views
Forecasting Prediction Accuracy
Out of 4 error paramters which one is best for evaluating prediction accuracy?
Average error
Mean absolute error
Mean squared error
Mean absolute % error
why?
2
votes
1answer
337 views
Mean Absolute Error and Data Distribution
I use a memory-based learning model to predict human scores in a [0, 10] range (quiz results). As a forecast error metric I use Mean Absolute Error. I was wondering what is the relation between MAE ...
2
votes
1answer
849 views
Measuring forecast accuracy
We're forecasting sales data for one of our clients on a weekly basis. Sales is forecasted for each organizational unit.
The sales data is forecasted via different algorithms and/or algorithm ...
4
votes
1answer
11k views
Which is the best accuracy measuring criteria among rmse, mae & mape?
I have created training set and test set from my data. Then I performed auto.arima() and ets() in R on the training set to predict one-step ahead forecasts. These were then compared with the test set ...
53
votes
3answers
45k views
Mean absolute error OR root mean squared error?
Why use Root Mean Squared Error (RMSE) instead of Mean Absolute Error (MAE)??
Hi
I've been investigating the error generated in a calculation - I initially calculated the error as a Root Mean ...
14
votes
3answers
34k views
Why use a certain measure of forecast error (e.g. MAD) as opposed to another (e.g. MSE)?
MAD = Mean Absolute Deviation
MSE = Mean Squared Error
I've seen suggestions from various places that MSE is used despite some undesirable qualities (e.g. http://www.stat.nus.edu.sg/~staxyc/T12.pdf, ...
1
vote
0answers
1k views
Relationship between $R^2$ and MAE in forecasting
I have the following linear model based on multivariate timeseries:
...
