# Questions tagged [accuracy]

Accuracy of an estimator is the degree of closeness of the estimates to the true value. For a classifier, accuracy is the proportion of correct classifications. (This second usage is not good practice. See the tag wiki for a link to further information.)

549 questions
Filter by
Sorted by
Tagged with
40k views

### Why is accuracy not the best measure for assessing classification models?

This is a general question that was asked indirectly multiple times in here, but it lacks a single authoritative answer. It would be great to have a detailed answer to this for the reference. ...
10k views

### What is the best way to remember the difference between sensitivity, specificity, precision, accuracy, and recall?

Despite having seen these terms 502847894789 times, I cannot for the life of me remember the difference between sensitivity, specificity, precision, accuracy, and recall. They're pretty simple ...
37k views

### Training a decision tree against unbalanced data

I'm new to data mining and I'm trying to train a decision tree against a data set which is highly unbalanced. However, I'm having problems with poor predictive accuracy. The data consists of students ...
17k views

### Why do I get a 100% accuracy decision tree?

I'm getting a 100% accuracy for my decision tree. What am I doing wrong? This is my code: ...
19k views

### What are the shortcomings of the Mean Absolute Percentage Error (MAPE)?

The Mean Absolute Percentage Error (mape) is a common accuracy or error measure for time series or other predictions, $$\text{MAPE} = \frac{100}{n}\sum_{t=1}^n\frac{|A_t-F_t|}{A_t}\%,$$ where $A_t$ ...
6k views

### Is an overfitted model necessarily useless?

Assume that a model has 100% accuracy on the training data, but 70% accuracy on the test data. Is the following argument true about this model? It is obvious that this is an overfitted model. The ...
16k views

### F1/Dice-Score vs IoU

I was confused about the differences between the F1 score, Dice score and IoU (intersection over union). By now I found out that F1 and Dice mean the same thing (right?) and IoU has a very similar ...
21k views

### Interpretation of mean absolute scaled error (MASE)

Mean absolute scaled error (MASE) is a measure of forecast accuracy proposed by Koehler & Hyndman (2006). $$MASE=\frac{MAE}{MAE_{in-sample, \, naive}}$$ where $MAE$ is the mean absolute error ...
1k views

### How can we judge the accuracy of Nate Silver's predictions?

Firstly, he gives probability of outcomes. So, for example, his predictions for the U.S. election is currently 82% Clinton vs 18% Trump. Now, even if Trump wins, how do I know that it wasn't just the ...
12k views

### How is the confusion matrix reported from K-fold cross-validation?

Suppose I do K-fold cross-validation with K=10 folds. There will be one confusion matrix for each fold. When reporting the results, should I calculate what is the average confusion matrix, or just sum ...
2k views

### Are there parameters where a biased estimator is considered “better” than the unbiased estimator? [duplicate]

A perfect estimator would be accurate (unbiased) and precise (good estimation even with small samples). I never really thought of the question of precision but only the one of accuracy (as I did in ...
3k views

### If “Standard error” and “Confidence intervals” measure precision of measurement, then what are the measurements of accuracy?

In book "Biostatistics for dummies" in page 40 I read: The standard error (abbreviated SE) is one way to indicate how precise your estimate or measurement of something is. and Confidence ...
17k views

### Good accuracy despite high loss value

During the training of a simple neural network binary classifier I get an high loss value, using cross-entropy. Despite this, accuracy's value on validation set holds quite good. Does it have some ...
6k views

### Accuracy vs. area under the ROC curve

I constructed an ROC curve for a diagnostic system. The area under the curve was then non-parametrically estimated to be AUC = 0.89. When I tried to calculate the accuracy at the optimum threshold ...
15k views

### What are the differences between AUC and F1-score?

F1-score is the harmonic mean of precision and recall. The y-axis of recall is true positive rate (which is also recall). So, sometime classifiers can have low recall but very high AUC, what that ...
19k views

### Is accuracy = 1- test error rate

Apologies if this is a very obvious question, but I have been reading various posts and can't seem to find a good confirmation. In the case of classification, is a classifier's accuracy = 1- test ...
27k views

### Is the Dice coefficient the same as accuracy?

I come across the Dice coefficient for volume similarity (https://en.wikipedia.org/wiki/S%C3%B8rensen%E2%80%93Dice_coefficient) and accuracy (https://en.wikipedia.org/wiki/Accuracy_and_precision). ...
2k views

### Is my model any good, based on the diagnostic metric ($R^2$/ AUC/ accuracy/ RMSE etc.) value?

I've fitted my model and am trying to understand whether it's any good. I've calculated the recommended metrics to assess it ($R^2$/ AUC / accuracy / prediction error / etc) but do not know how to ...
1k views

### Is accuracy an improper scoring rule in a binary classification setting?

I have recently been learning about proper scoring rules for probabilistic classifiers. Several threads on this website have made a point of emphasizing that accuracy is an improper scoring rule and ...
12k views

### How to determine the accuracy of regression? Which measure should be used?

I have problem with defining the unit of accuracy in a regression task. In classification tasks is easy to calculate sensitivity or specificity of classifier because output is always binary {correct ...
1k views

### Why is feature selection important, for classification tasks?

I'm learning about feature selection. I can see why it would be important and useful, for model-building. But let's focus on supervised learning (classification) tasks. Why is feature selection ...
295 views

### Voting system that uses accuracy of each voter and the associated uncertainty

Let's say, we have simple "yes/no" question that we want to know answer to. And there are N people "voting" for correct answer. Every voter has a history - list of 1's and 0's, showing whether they ...
11k views

### Is f-measure synonymous with accuracy?

I understand that f-measure (based on precision and recall) is an estimate of how accurate a classifier is. Also, f-measure is favored over accuracy when we have an unbalanced dataset. I have a simple ...
3k views

### Evaluation of classifiers: learning curves vs ROC curves

I would like to compare 2 different classifiers for a multiclass text classification problem that use large training datasets. I am doubting whether I should use ROC curves or learning curves to ...
20k views

### Forecast accuracy calculation

We are using STL (R implementation) for forecasting time series data. Every day we run daily forecasts. We would like to compare forecast values with real values and identify average deviation. For ...
38k views

### Interpretation of AIC value

Typical values of AIC that I have seen for logistic models are in thousands, at least hundreds. e.g. On http://www.r-bloggers.com/how-to-perform-a-logistic-regression-in-r/ the AIC is 727.39 While ...
874 views

### Example when using accuracy as an outcome measure will lead to a wrong conclusion

I am looking into various performance measures for predictive models. A lot was written about problems of using accuracy, instead of something more continuous to evaluate model performance. Frank ...
62 views

### How to dissuade laypeople from drawing inaccurate conclusions about their data?

I work as a data analyst, primarily in SQL, providing operations data to internal customers. I rarely do statistical analysis. Recently, internal customers have been coming to me with data from ...
2k views

### Why not using the R squared to measure forecast accuracy?

Why in literature usually the common accuracy measures like MAD, MSE, RMSE, MAPE ... are used. Why not using the $R^2$ (coefficient of determination)? I was thinking about the difference: By using ...
372 views

### Should predictive accuracy or, alternatively, minimizing the MSE, be reconsidered?

Ever since Breiman, maximizing predictive accuracy has become a predictive modeling gold standard, of sorts. That it has evolved to this status is understandable: it can be "optimized," is easily ...
223 views

### How to know if a statistic prediction is right?

The weather forecast predict the probability of rain or not for some day. If I could repeat the same day many times I could count how many times does it rain or not, so I could compare with the ...
10k views

### Time series forecasting accuracy measures: MAPE and MASE

We come to this toy example showing MAPE and MASE are not consistent when measuring forecasting accuracy. Data consist of 100 white noise and 100 $AR(1)$ time series with length $N=500$, mean $\mu=1$ ...
3k views

### Why the sum of true positive and false positive does not have to be equal to one?

While reading this question above, I got confused about the sum of true positive and false positive. If an aircraft is present in a certain area, a radar detects it and generates an alarm signal ...
5k views

### Should I get 100% classification accuracy on training data?

I've been getting inconsistent results with a binary classification problem I'm trying to solve using a linear classifier and a custom feature extraction pipeline, and decided to do a quick check of ...
2k views

### What's the measure to assess the binary classification accuracy for imbalanced data?

Now I have binary classification problem with positive samples roughly 100 times the number of negative samples. In this case the normal accuracy measure (predict == label) is not a good measure. What ...
25k views

### Calculating MAPE [closed]

Is the below calculation of the Mean Absolute Percentage Error MAPE correct? I've included a workable example, but really the lines in question are these: ...
4k views

### Accuracy of model lower than no-information rate?

I have a dataset with predicted variable having two classes; true and false. 99.99% of the values are with false class. In this case, no-information rate is 99.99%. So, any model that I build needs to ...
7k views

### Diebold-Mariano test for predictive accuracy

I am using the Diebold-Mariano test in the forecast package of R to test the predictive accuracy. In particular, I want to ...
397 views

### Is the objective to beat a random classifier when the data set is skewed using PR curves?

I have a testing data set where 1/3 of the observations are class-1 objects and the remainder class-0. Hence, the data set is skewed (skewed classifier), literature suggests that if the data set is ...
218 views

### Should OOB (Out Of Bag) error be less than a Test set error in Random Forests?

I am using the book, "An introduction to statistical learning with applications in R" and reading the section on using OOB to estimate the model error for Random Forests. The graph seems to suggest ...
2k views

### Loss vs. Classification Accuracy in applied problems

In practical problems, where we want to for instance predict if a subject has a certain disease or not, we usually take classification accuracy as a measure which has the straightforward ...
2k views

### Forecast accuracy metric that involves prediction intervals

I'm in the process of generating a time series forecast for a company's product revenue and am looking for some way to show accuracy over time - e.g. after say 6 months they want to see how the actual ...
398 views

### How to select the best classifier in classification task?

Suppose in the task of classification, I want to select the best classifier. How to do this? My idea is to select the classifier which gives the highest classification accuracy using the test data ...
9k views

### AUC and Balanced accuracy in R Modelling

Can someone please explain the difference between AUC(Area under curve) and balanced accuracy in R? For eg: In decision tree modelling I got the, AUC : 0.91 balanced accuracy : 0.72 please explain ...
605 views

### How to transform an accuracy distribution for a violin plot

I am trying to find the best way to visualize different distributions of accuracy. Accuracy here is a value in the interval [0,1], 0 meaning not accurate, 1 maximum accuracy. I have different methods ...
3k views

### validation/training accuracy and overfitting

If we randomly split the data into training data and validation data, and assume the training data and validation data have similar "distributions", i.e. they are both good representations of the ...
3k views

### Evaluating relative accuracy/error for continuous value prediction (and assessing relative average difference/error)

I was trying to understand what are good (or standard) way of evaluating relative accuracy for continuous data. Say for example, say I have some statistical algorithm S that outputs some real number ...
116 views

### How do I call a forecast that is both accurate and precise?

How do I call a forecast (more precisely, a forecasting rule) that is both accurate and precise? Is there a word that expresses both properties combined? I do not mean the forecasting rule is perfect,...
I'm doing an independent study in Bayesian Statistics following some chapters from BDA3. When solving the first question from Ch 10 I got stuck. It says: [If] a scalar variable $\theta$ is ...