Stack Exchange Network

Stack Exchange network consists of 175 Q&A communities including Stack Overflow, the largest, most trusted online community for developers to learn, share their knowledge, and build their careers.

Visit Stack Exchange

Questions tagged [accuracy]

Accuracy of an estimator is the degree of closeness of the estimates to the true value. For a classifier, accuracy is the proportion of correct classifications. (This second usage is not good practice. See the tag wiki for a link to further information.)

0
votes
1answer
11 views

Expected Misclustering rate

I am reading this paper on minimax clustering error rates on high-dimensional Gaussian mixtures. The authors define a metric for expected misclustering rate as follows: For a two-component ...
0
votes
0answers
10 views

Reference Request: Overall and Average Accuracy [duplicate]

I am looking for a source to cite on the definitions of overall and average accuracy. I have found many informal sources online, including here on CVSE, but the papers and textbooks I have found seem ...
0
votes
1answer
10 views

Accuracy and F-mesure for imbalanced datasets

I have 10 imbalanced datasets. Classes are : 1, 2, 3, ..., 10,11,12. I used as evaluation metrics for my datasets accuracy and F-measure. The F-messure of each class in each dataset is as below: Is ...
0
votes
1answer
18 views

Measuring predictive accuracy of an ordinal outcome when the predictor is continuous

I'm predicting a response measured by a 5-point likert scale from a model that produces continuous predictions of the same variable. For example: ...
0
votes
0answers
16 views

what test is suitble to predict seen and unseen data

In my model, I want to predict both seen and unseen data and get the result that maximizes the accuracy. The problem is that in some cases seen data are well predicted since the model is overfitting, ...
1
vote
0answers
19 views

Ideal score of a model on training and cross validation data

The question is little bit broad, but I could not find any concrete explanation anywhere, hence decided to ask the experts here. I have trained a classifier model for binary classification task. Now ...
0
votes
0answers
13 views

Do I choose the logistic regression model with the better F1 statistic on the validation set or the lower AIC?

I am deciding whether to keep an interaction term in my logistic regression model. If I keep it in, the AIC of the model improves. If I leave it out, I get a better F1 score with my validation set. ...
2
votes
1answer
32 views

Balanced datasets are almost all predicted negative

Problem I am trying to do sentiment analysis using pretrained word vectors GloVe, which is essentially a look-up table that maps word to a fix-dimension vector. Since GloVe is initially designed to ...
1
vote
0answers
11 views

Downsampling, AUROC and accuracy equal

I am using downsampling to create perfectly balanced classes in my target feature. I have found that accuracy is exactly equal to the AUROC score. I was thinking that this is because I've used ...
0
votes
0answers
30 views

Is there a name for “normalized accuracy” as a statistic?

In short: I'm using a statistic representing the "normalized accuracy" of a confusion matrix. Is there a formal term for this? $$\text{normacc} = \frac{\text{acc}-\text{thacc}}{1-\text{thacc}}$$ ...
0
votes
1answer
15 views

Correct calculation of repeated cross-validation classification metrics

We can obtain a resampled estimate of training set classification accuracy from caret::confusionMatrix.train(model) e.g., ...
0
votes
1answer
13 views

Does the threshold value of a logistic regression hypothesis has an effect on the accuracy?

It is true that the threshold value of a logistic regression hypothesis has an effect on the Precision/Recall metrics. Suppose you have trained a logistic regression classifier which is outputting $...
0
votes
0answers
23 views

How to compare Accuracy of two RandomForest models? (Chi Sqr or Cohen's H?)

I've got two dataset which have exactly the same structure (15 features, 1 class variable with 7 categories) and roughly the same amount of observations). I trained a Random Forest with the full ...
1
vote
2answers
50 views

Is the accuracy equal if the true positive and false positive rates are equal between two groups?

I was reading the paper "Equality of Opportunity in Supervised Learning" (link). In that paper there is a feature $A \in \{0,1\}$ and a binary outcome $y \in \{0,1\}$. The population is divided into ...
0
votes
0answers
12 views

What type of accuracy should I reported in research paper?

I have read some research papers on the classification task of deep learning, and now I am doing my own. After investigating some research paper which also provided the source code for reproducing ...
2
votes
1answer
55 views

Can balanced accuracy be higher than accuracy?

I have classification tree where the balanced accuracy of the test set is higher than the normal accuracy. I thought balanced accuracy can only have at his maximum the same value as the accuracy not ...
1
vote
0answers
14 views

How to compare predicted accuracy and actual accuracy? [duplicate]

Consider a classifier that, given an input vector ${\bf x}$ outputs both a prediction $y'$ whose accuracy ($a \in \{0, 1\}$) can be measured, as well as a predicted accuracy that corresponds to the ...
1
vote
0answers
25 views

Why we always get different accuracy for a different number of training our model? [duplicate]

As the question says for example if I train my neural network (with 2 layers) model the first time it gives me a score $A \in \mathbb{R}$ and when I train the same model again it gives a different ...
1
vote
1answer
25 views

Discrimination (pseudo) $R^2$ vs. C-index

In the context of binary logistic regression. Both pseudo $R^2$ and C-index measures the discrimination of the model. But why do you need both ? can you gain something from one but not from the other?
2
votes
1answer
29 views

Constant validation loss and increasing validation accuracy

I am training a fully convolutional network. The loss is decreasing whilst the validation loss stays mostly where it is. There is some variance in the validation loss. I thought it might overfits, ...
0
votes
0answers
22 views

How can the balanced accurcay be bigger than the normal accuracy in unbalanced test data? [duplicate]

I constructed two binary classification tree's on two different training set's that i balanced with oversampling and undersampling. The test set is still unbalanced. After that i computetd the ...
4
votes
1answer
143 views

What is the difference between an accuracy measure and an error metric?

The two concepts are distinct in measure theory. Nonetheless, moving out from measure theory, the two terms are often used interchangeably. To most forecasters, especially forecast practitioners, they ...
0
votes
0answers
12 views

Best practice for presenting classifier accuracy using cross-validation?

I have a classifier whose classification accuracy I must present. Of course, I could just do e.g. a single 90/10 train/test split and report the test accuracy. However, my dataset is fairly small, so ...
0
votes
0answers
29 views

Neural Network Accuracy Bouncing Around and Never Going Over 50% Accuracy [Not Duplicate] [duplicate]

My NN accuracy is bouncing between .29 and .37. Sometimes it starts at .5, but then decreases as it continues. The loss also bounces around, decreasing, increasing, and generally staying around 1. The ...
0
votes
3answers
91 views

Machine learning without test and validation data

All mainstream machine learning approaches I've seen depend on a test and usually a validation dataset to measure model accuracy during and after training. This seems like it uses up quite a lot of ...
0
votes
1answer
27 views

Is it possible to estimate accuracy, precision and recall with the given data?

Background: I talked to my friend today and according to herm(him/her) I can calculate precision, recall and accuracy with the current information. Total instances T: 19,532. Instances belonging to ...
1
vote
1answer
25 views

Error measure for 3D-field-comparison - meaning of mean/median

Short Version I have to compare two vectors of predictions (from different methods) against one vector of measurements to find out which prediction performs better. Note that this is not a ...
1
vote
2answers
51 views

accuracy of a regression prediction model

I developed two prediction models using non-linear regression analysis to predict a set of values using sigmoidal and power functions. I was wondering how I can evaluate the accuracy of these ...
0
votes
1answer
77 views

random forest regression predicts “opposite”

I have a dataset with 70 features, which are continuous measures and are interrelated but not highly correlated ($|\rho| <.5$. I have several outcomes, which are each integer values ranging from 0-...
2
votes
1answer
15 views

Precision vs. Accuracy when talking about MSE

This is more of a semantic question. I'm working on translating a work from French to English related to statistics. In French, there is only 1 word as far as I can tell to describe both bias and ...
1
vote
1answer
28 views

Which value of accuracy or balanced accuracy is enough?

I constructed a classification tree and want validate the out of sample performance. I read that the accuracy or the balanced accuracy must at least higher than the no information rate. By the no ...
0
votes
0answers
20 views

Knn Classification on iris dataset

I'm following along https://rpubs.com/Drmadhu/IRISclassification to understand Knn classification. Here's the code I have: library(FNN) iris.sample<-sample.int(n=nrow(irisdat),size=floor(0.75*...
2
votes
1answer
128 views

Learning a quadratic function using TensorFlow/Keras

Heads up: I'm not sure if this is the best place to post this question, so let me know if there is somewhere better suited. I am trying to train a simple neural network to learn a simple quadratic ...
2
votes
1answer
34 views

Which approach should be used to compare two different measurement techniques of same samples?

I have individually measured failure forces of 8 materials and those recorded with A method and B method in same time: 8 results in each method, A=8 and B=8. The range of data of both measurement ...
1
vote
1answer
10 views

Is it appropriate to use a confusion matrix for frequency data?

I am testing the accuracy of a machine learning approach that counts cars in images. I have both a predicted dataset and a "real" dataset that was generated by a human. For example, this is what my ...
0
votes
0answers
48 views

Accuracy score or AUC extracted from Gradient Boosting Classifier of scikit-learn? [duplicate]

I'm working on developing a predictive model for a binary classification problem related to biomedical applications (need a really high and promising accuracy). I'm training on my training dataset and ...
5
votes
1answer
95 views

Should OOB (Out Of Bag) error be less than a Test set error in Random Forests?

I am using the book, "An introduction to statistical learning with applications in R" and reading the section on using OOB to estimate the model error for Random Forests. The graph seems to suggest ...
0
votes
0answers
15 views

Acceptable Accuracy, Precision, Sensitivity, and Specificity Thresholds [duplicate]

Are there general rule of thumbs for acceptable accuracy, precision, sensitivity, and specificity values/thresholds in classification? I would imagine that this depends on different applications. I ...
0
votes
1answer
49 views

Mean Absolute Scaled Error implementation on multistep time series forecast

The formula for MASE can be found here: https://en.wikipedia.org/wiki/Mean_absolute_scaled_error I am building a multi-step time series forecaster and I want to use MASE as a measure of prediction ...
0
votes
0answers
34 views

How can MASE (Mean Absolute Scaled Error) score value be interpreted for non time series data?

If I have used MASE to calculate non time-series data error (as described by Dr. Rob Hyndman here), how can I know if the score received is good or not? Since it is not a time-series, a random walk ...
1
vote
0answers
10 views

What measures of an ML algorithm's 'accuracy' are mostly consistent as the number of classes to predict into varies?

For a research project, I've got a bunch (N=507) of 20-second VR tracking data clips (6DOF x head and hands), each from a different participant. My goal is to predict the participant using a small ...
3
votes
1answer
45 views

Including Collider Variables in Prediction

When the goal is to estimate a causal association between X and Y in the regression framework, one should not condition on (include as covariates) collider variables (common causes of both X and Y) ...
0
votes
1answer
199 views

why the accuracy of my CNN decreasing after some epochs?

at high accuracy, after some epochs the accuracy as well as validation accuracy is decreasing and got stuck after few more epochs. i dont understand why this happened. does more epochs at some point ...
0
votes
0answers
14 views

increasing accuracy of binary logistic regression by reducing type II error?

I am to use binary logistic regression predict a deadly disease from 109 cases out of 385 patients. If during the preliminary diagnosis all patients were sent to the expert doctor for secondary ...
0
votes
1answer
32 views

accuracy and precision in regression vs classification

Are accuracy and precision the same things in regression and classification? In regression: accuracy is bias, and precision is inverse of variance. In classification: accuracy is correct prediction ...
0
votes
1answer
42 views

improving accuracy of classification model

I have data with 95 numeric variables and 5 categorical variables. My Y has 2 values. I built a decision tree and my accuracy was 81.8%. Then I created 3 new variables as follows. It improved accuracy ...
0
votes
0answers
30 views

Machine Learning: Model doesn´t recognize letters but has 80% accuracy

I have build a model to classify numbers and characters on Images. I trained it on the Chars74K dataset and in training it has 80% validation accuracy. I just use the number and uppercase characters ...
0
votes
0answers
371 views

Leave One Out Cross-Validation in Python

For me is not clear the way to implement LOOCV in Python, I have the next Python scripts: 1) ACC = 76.92 % ...
0
votes
0answers
21 views

False negative probability of cross-correlation

Suppose $D$ is a string of length $L$ where its values are uniformly distributed in $[-a,a]$. Also, all values of $D$ are independent (i.i.d). $X$ is a noisy version of $D$ in this way: $X=D+N$ where ...
0
votes
0answers
24 views

Interpreting accuracy values in ARIMA

How to interpet each measuring accuracy and how will i know if its accurate measure in the model