# Questions tagged [accuracy]

Accuracy of an estimator is the degree of closeness of the estimates to the true value. For a classifier, accuracy is the proportion of correct classifications. (This second usage is not good practice. See the tag wiki for a link to further information.)

164 questions with no upvoted or accepted answers
Filter by
Sorted by
Tagged with
127 views

### Question 10.9 from Bayesian Data Analysis, what does accuracy mean here?

I'm doing an independent study in Bayesian Statistics following some chapters from BDA3. When solving the first question from Ch 10 I got stuck. It says: [If] a scalar variable $\theta$ is ...
230 views

### Can F1-Score be higher than accuracy?

I'm using sklearn's confusion_matrix and classification_report methods to compute the ...
172 views

### Accuracy of a polygon fitting algorithm

I have a polygon $P$ which I have to fit with a rectangle $R$, as shown in the figure below. This is done by varying the dimension of the rectangle, and choosing the dimension that best fits the ...
32 views

### Precision vs. Accuracy when talking about MSE

This is more of a semantic question. I'm working on translating a work from French to English related to statistics. In French, there is only 1 word as far as I can tell to describe both bias and ...
39 views

### Which approach should be used to compare two different measurement techniques of same samples?

I have individually measured failure forces of 8 materials and those recorded with A method and B method in same time: 8 results in each method, A=8 and B=8. The range of data of both measurement ...
39 views

### With test accuracy being equal, is it better to have lower training accuracy?

Suppose we train two models on a training set, and then test them both on the training set itself, and on a test set. We have some accuracy metric we're using to evaluate them. Both models score ...
352 views

### Calculation of accuracy (and Cohen's kappa) using sensitivity, specificity, positive and negative predictive values

I read How to calculate specificity from accuracy and sensitivity, but I have two diagnostic performance measures more. Please correct me if I am wrong: if Sensitivity=TP/(TP+FN) Specificity=TN/(TN+...
128 views

### Classification model accuracy, roc auc score, f1 score 100%

I am working on a binary classification problem. I have split the train set and when I evaluate the model on the validation set all metrics are 100% which is unrealistic considering that I haven't ...
303 views

### Accuracy metric for comparing Time Series models?

I'm writing a blog post on forecasting time series with autoregression. In it, I compare the performance of SLR, ARIMA, and SARIMAX on forecasting the number of Home Sales in Seattle (see below). All ...
196 views

### Difference between cumulative gains chart and cumulative accuracy profile for binary classifier

I am confused about the following: Here I find the definition of cumulative gains chart as the plot of x-axis: the rate of predicted positive y-axis: true positive rate It turns out that we can e....
101 views

### Binary Classification: good at predicting negative class but bad at predicting positive class

I used many different algorithms on a data set for binary classification. I used kNN, SVM radial, ANN, random forest, Gaussian process classification, etc. Every algorithm is well tuned in R using ...
290 views

### Is it always better to use F1-Score than Accuracy as performance metric?

During reading papers about Machine Learning I always find researchers using accuracy as their sole performance metric. However, a high accuracy alone proves nothing when the amount of false positives ...
118 views

### Comparing two split criteria for decision trees

Consider a split criterion for decision trees, which favors splits resulting in groups with as evenly distributed classes as possible. What will be the effect on the resulting decision trees compared ...
114 views

### Does It make sense to interprete variables with low accuracy but high R squared in Random Forest?

I did random forest analysis and I obtained two graphs. In the first one (left hand side) we can see the measure of importance of each predictor and on the second one (right hand side) is the ...
202 views

### Rleation between L2 loss and accuracy

Given a classification problem with k classes, suppose a model outputting a probability distribution over the classes that uses some gradient-based learning method is used (yes, in my case it's neural ...
77 views

### Classifying users in one of two groups

I've a - probably beginner level - question about classifying. My goal is to classify users as UG1 or UG2, based on some specific characteristics of them. So, what I've done so far is: I have two ...
145 views

### How would you interpret decreasing cost but increasing training and validation error during epochs?

I’m training a fully convolution network with final layer of global sum pooling and no intermediate pooling in the network which I already test on global average pooling and it converged very well. ...
129 views

### How to compare the forecast accuracy of two models when the data has unbalanced panel structure?

I am comparing the earnings forecast accuracy of two models (model 1 and model 2). The data (firm-year level data) have an unbalanced panel structure since firms have varying lifetime and time to be ...
3k views

### How to threshold multiclass probability prediction to get confusion matrix?

Lets say my multinomial logistic regression predict that a chance of a sample belonging to a each class is A=0.6, B=0.3, C=0.1 How do I threshold this values to get just binary prediction of a sample ...
168 views

### Calculating NPV from precision, recall and CA

I am trying to simulate predictions of a binary classifier, given its precision, recall, classification accuracy ($CA$) and the actual classes of the instances. I.e. I have instances and I know what ...
89 views

### Variable Importance's reliability when model accuracy is low

I am trying to predict the termination events for a company. I also want to rank the most important variables that lead to termination. I ran a decision tree model and have the following questions: ...
72 views

### biological sample size when instrument accuracy is weak?

I'm using a porometer to measure the transpiration of grape leaves. Unfortunately, the instrument's accuracy is 10% and it has a relatively slow (60 seconds) round trip time to produce readings when ...
745 views

### Is using Rpart with unbalanced data a good idea?

I have a rather unbalanced data set and want to use rpart to build a classification tree. After building the full tree, I prune it back using the 1-SE rule. On average, only 1-2 splits are suggested. ...
314 views

### Interpreting Elastic Net Coefficients and Accuracy Testing

I have a code for an elastic net model which is predicting the best lambda to use. With this, it provides coefficients for those values. I wanted to see how accurate those coefficients are that it ...
1k views

### Test MAPE < Train MAPE using auto.arima()

I am trying to build a forecasting model for the passenger vehicles registrations in a given country, and I wanted to use $auto.arima$ function from the $forecast$ package to estimate a simple ARIMA ...
74 views

### How do you report percent error with limited precision?

Let's say you have an experimental data point with limited precision, such as 0.67 ± 0.1, and a known actual value such as 0.72. What is the standard way to report the percent error in this case, ...
585 views

### Is there an effect size for Kappa's?

I am staring a project on comparing standard ways of creating a classifier with some heuristic methods. The heuristic methods should result in a faster training for the classifier but should result in ...
853 views

### How to calculate accuracy of each feature

I read some paper (* about predicting user retention in StumbledUpon) and saw the authors provide a list of features with accuracy of each feature with the following explanation: As decision ...
62 views

### How do you combine the error of a device with the standard deviation of the readings from it?

I have recorded 3 readings of pressure at a set flow rate for about 8 different flow rates. I know the accuracy of my manometer, and can calculate stddev from the 3 points for each. How do you combine ...
29 views

### Understanding Accuracy, Recall and IoU

Working on an image segmenetation problem, I've tackled the following scenario repeated on different images: High Recall and Accuracy (around 99%) Low IoU (around 60%) How is that possible? Recall ...
21 views

### How can I define a accuracy measure for word2vec predictions

I have a data set consisting of tags and some classes.I'm suppose to find the nearest class to each set of tags with Word2vec embeddings and cosine similarity.Each set of tags have multiple classes ...
21 views

### Ideal score of a model on training and cross validation data

The question is little bit broad, but I could not find any concrete explanation anywhere, hence decided to ask the experts here. I have trained a classifier model for binary classification task. Now ...
12 views

### Downsampling, AUROC and accuracy equal

I am using downsampling to create perfectly balanced classes in my target feature. I have found that accuracy is exactly equal to the AUROC score. I was thinking that this is because I've used ...
10 views

### What measures of an ML algorithm's 'accuracy' are mostly consistent as the number of classes to predict into varies?

For a research project, I've got a bunch (N=507) of 20-second VR tracking data clips (6DOF x head and hands), each from a different participant. My goal is to predict the participant using a small ...
91 views

### Measure of accuracy for a Bayesian model

I am reading Statistical Rethinking (Section 6.2.1.2). The topic of this section is measuring accuracy for a Bayesian model, i.e. accuracy of the model of predicting correctly an outcome. The ...
101 views

### Naive Bayes and kNN accuracy

Assume that a large number of binary features are added to a dataset with two class labels c1 and c2, such that for each added feature f, the class conditional probability P(f = 0|c1) = P(f = 0|c2). ...
42 views

### How to know if your decision tree model has overfitting or not?

I am using the DecisionTreeClassifier() of python and I am changing some tuning parameters to understand if my model has overfitting or no because when I exectue the following code without any ...
53 views

### How to correctly compare the accuracy of different forecasting methods using bootstrapping with time series forecasting

I am currently working on a forecasting project and I have tried several different models to forecast with. Having trained and tuned my models I want to pick which model works best for each time ...
31 views

### Boosted decision trees: in which situations are “deep” decision trees performing better?

The general idea of boosted decision trees is to use very simple trees in the following manner (simplified, for intuition only): start with a simple tree, fit another simple tree on the residuals, ...
37 views

### Compare accuracy between tools using k-fold cross validation, each tool is tested with different k values

I'm working on a new way to do the classification in a supervised way and I want to compare its accuracy to some related works. These works are using the same data set and they are testing their ...
164 views

### Confusions about Pesaran & Timmermann test (2009 version)

I am learning about Pesaran Timmermann test (2009) for directional accuracy and I have troubles understanding its formula. Here I use notation and arrangement used in Pönkä (2017): PT=(T-1)(S^{-1}_{...
54 views

### Metric to evaluate accuracy of time series decomposition?

I am trying to de-seasonalize a large number of business sales data. They all come from the same industry and follow idiosyncratic seasonality patterns. The number of businesses is quite large, and I ...
34 views

### How to make really bad results from a machine learning model better by reversing predictions

I trained a classification model on some data with two classes and have really low accuracy. I have a false-positive rate of 86 % for both classes I am trying to predict. I was wondering if I could ...
128 views

### Alternative measurement for Cross Entropy Loss than accuracy when using soft labeling?

Suppose one has multi-class ($n>2$) data, where class labels are soft, e.g. two samples of for 3-classed data might look like: ...
429 views

### Recall equals to accuracy but different to precision

I've read this question, and basically I'm having the same issue. I'm dealing with a binary classification problem. I'm calculating the precision, recall and f1 using ...
100 views

### Can i use Diebold Mariano test for comparison of 2 models across multiple time series?

I have 2 models (for simplicity, let's call them AR(1) and MA(1)) each making 1 day ahead forecasts of time series. If I had only 1 time series I would just use ...
115 views

### Comparing two models with statistical testing

I am working on a neural network architecture to tackle a problem and I want to compare my model to another model used in literature. I use k-fold crossvalidation to get a more unbiased accuracy and ...
57 views

### Can we increase the accuracy of a classifier using sketches?

I am using a sketch technique to improve the memory of a standard classifier (naive Bayes) with data streams. The sketch technique is composed of a sketch table (hash table) means the true values can ...