Questions tagged [accuracy]
Accuracy of an estimator is the degree of closeness of the estimates to the true value. For a classifier, accuracy is the proportion of correct classifications. (This second usage is not good practice. See the tag wiki for a link to further information.)
0
votes
1answer
11 views
Expected Misclustering rate
I am reading this paper on minimax clustering error rates on high-dimensional Gaussian mixtures. The authors define a metric for expected misclustering rate as follows:
For a two-component ...
0
votes
0answers
10 views
Reference Request: Overall and Average Accuracy [duplicate]
I am looking for a source to cite on the definitions of overall and average accuracy. I have found many informal sources online, including here on CVSE, but the papers and textbooks I have found seem ...
0
votes
1answer
10 views
Accuracy and F-mesure for imbalanced datasets
I have 10 imbalanced datasets. Classes are : 1, 2, 3, ..., 10,11,12.
I used as evaluation metrics for my datasets accuracy and F-measure.
The F-messure of each class in each dataset is as below:
Is ...
0
votes
1answer
18 views
Measuring predictive accuracy of an ordinal outcome when the predictor is continuous
I'm predicting a response measured by a 5-point likert scale from a model that produces continuous predictions of the same variable.
For example:
...
0
votes
0answers
16 views
what test is suitble to predict seen and unseen data
In my model, I want to predict both seen and unseen data and get the result that maximizes the accuracy. The problem is that in some cases seen data are well predicted since the model is overfitting, ...
1
vote
0answers
19 views
Ideal score of a model on training and cross validation data
The question is little bit broad, but I could not find any concrete explanation anywhere, hence decided to ask the experts here.
I have trained a classifier model for binary classification task. Now ...
0
votes
0answers
13 views
Do I choose the logistic regression model with the better F1 statistic on the validation set or the lower AIC?
I am deciding whether to keep an interaction term in my logistic regression model. If I keep it in, the AIC of the model improves. If I leave it out, I get a better F1 score with my validation set. ...
2
votes
1answer
32 views
Balanced datasets are almost all predicted negative
Problem
I am trying to do sentiment analysis using pretrained word vectors GloVe, which is essentially a look-up table that maps word to a fix-dimension vector. Since GloVe is initially designed to ...
1
vote
0answers
11 views
Downsampling, AUROC and accuracy equal
I am using downsampling to create perfectly balanced classes in my target feature. I have found that accuracy is exactly equal to the AUROC score. I was thinking that this is because I've used ...
0
votes
0answers
30 views
Is there a name for “normalized accuracy” as a statistic?
In short: I'm using a statistic representing the "normalized accuracy" of a confusion matrix. Is there a formal term for this?
$$\text{normacc} = \frac{\text{acc}-\text{thacc}}{1-\text{thacc}}$$
...
0
votes
1answer
15 views
Correct calculation of repeated cross-validation classification metrics
We can obtain a resampled estimate of training set classification accuracy from caret::confusionMatrix.train(model)
e.g.,
...
0
votes
1answer
13 views
Does the threshold value of a logistic regression hypothesis has an effect on the accuracy?
It is true that the threshold value of a logistic regression hypothesis has an effect on the Precision/Recall metrics.
Suppose you have trained a logistic regression classifier which is outputting $...
0
votes
0answers
23 views
How to compare Accuracy of two RandomForest models? (Chi Sqr or Cohen's H?)
I've got two dataset which have exactly the same structure (15 features, 1 class variable with 7 categories) and roughly the same amount of observations). I trained a Random Forest with the full ...
1
vote
2answers
50 views
Is the accuracy equal if the true positive and false positive rates are equal between two groups?
I was reading the paper "Equality of Opportunity in Supervised Learning" (link).
In that paper there is a feature $A \in \{0,1\}$ and a binary outcome $y \in \{0,1\}$. The population is divided into ...
0
votes
0answers
12 views
What type of accuracy should I reported in research paper?
I have read some research papers on the classification task of deep learning, and now I am doing my own. After investigating some research paper which also provided the source code for reproducing ...
2
votes
1answer
55 views
Can balanced accuracy be higher than accuracy?
I have classification tree where the balanced accuracy of the test set is higher than the normal accuracy. I thought balanced accuracy can only have at his maximum the same value as the accuracy not ...
1
vote
0answers
14 views
How to compare predicted accuracy and actual accuracy? [duplicate]
Consider a classifier that, given an input vector ${\bf x}$ outputs both a prediction $y'$ whose accuracy ($a \in \{0, 1\}$) can be measured, as well as a predicted accuracy that corresponds to the ...
1
vote
0answers
25 views
Why we always get different accuracy for a different number of training our model? [duplicate]
As the question says for example if I train my neural network (with 2 layers) model the first time it gives me a score $A \in \mathbb{R}$ and when I train the same model again it gives a different ...
1
vote
1answer
25 views
Discrimination (pseudo) $R^2$ vs. C-index
In the context of binary logistic regression. Both pseudo $R^2$ and C-index measures the discrimination of the model. But why do you need both ? can you gain something from one but not from the other?
2
votes
1answer
29 views
Constant validation loss and increasing validation accuracy
I am training a fully convolutional network. The loss is decreasing whilst the validation loss stays mostly where it is. There is some variance in the validation loss.
I thought it might overfits, ...
0
votes
0answers
22 views
How can the balanced accurcay be bigger than the normal accuracy in unbalanced test data? [duplicate]
I constructed two binary classification tree's on two different training set's that i balanced with oversampling and undersampling. The test set is still unbalanced.
After that i computetd the ...
4
votes
1answer
143 views
What is the difference between an accuracy measure and an error metric?
The two concepts are distinct in measure theory. Nonetheless, moving out from measure theory, the two terms are often used interchangeably. To most forecasters, especially forecast practitioners, they ...
0
votes
0answers
12 views
Best practice for presenting classifier accuracy using cross-validation?
I have a classifier whose classification accuracy I must present. Of course, I could just do e.g. a single 90/10 train/test split and report the test accuracy. However, my dataset is fairly small, so ...
0
votes
0answers
29 views
Neural Network Accuracy Bouncing Around and Never Going Over 50% Accuracy [Not Duplicate] [duplicate]
My NN accuracy is bouncing between .29 and .37. Sometimes it starts at .5, but then decreases as it continues. The loss also bounces around, decreasing, increasing, and generally staying around 1. The ...
0
votes
3answers
91 views
Machine learning without test and validation data
All mainstream machine learning approaches I've seen depend on a test and usually a validation dataset to measure model accuracy during and after training.
This seems like it uses up quite a lot of ...
0
votes
1answer
27 views
Is it possible to estimate accuracy, precision and recall with the given data?
Background: I talked to my friend today and according to herm(him/her) I can calculate precision, recall and accuracy with the current information.
Total instances T: 19,532.
Instances belonging to ...
1
vote
1answer
25 views
Error measure for 3D-field-comparison - meaning of mean/median
Short Version
I have to compare two vectors of predictions (from different methods) against one vector of measurements to find out which prediction performs better. Note that this is not a ...
1
vote
2answers
51 views
accuracy of a regression prediction model
I developed two prediction models using non-linear regression analysis to predict a set of values using sigmoidal and power functions. I was wondering how I can evaluate the accuracy of these ...
0
votes
1answer
77 views
random forest regression predicts “opposite”
I have a dataset with 70 features, which are continuous measures and are interrelated but not highly correlated ($|\rho| <.5$. I have several outcomes, which are each integer values ranging from 0-...
2
votes
1answer
15 views
Precision vs. Accuracy when talking about MSE
This is more of a semantic question. I'm working on translating a work from French to English related to statistics. In French, there is only 1 word as far as I can tell to describe both bias and ...
1
vote
1answer
28 views
Which value of accuracy or balanced accuracy is enough?
I constructed a classification tree and want validate the out of sample performance. I read that the accuracy or the balanced accuracy must at least higher than the no information rate. By the no ...
0
votes
0answers
20 views
Knn Classification on iris dataset
I'm following along https://rpubs.com/Drmadhu/IRISclassification to understand Knn classification.
Here's the code I have:
library(FNN)
iris.sample<-sample.int(n=nrow(irisdat),size=floor(0.75*...
2
votes
1answer
128 views
Learning a quadratic function using TensorFlow/Keras
Heads up: I'm not sure if this is the best place to post this question, so let me know if there is somewhere better suited.
I am trying to train a simple neural network to learn a simple quadratic ...
2
votes
1answer
34 views
Which approach should be used to compare two different measurement techniques of same samples?
I have individually measured failure forces of 8 materials and those recorded with A method and B method in same time: 8 results in each method, A=8 and B=8. The range of data of both measurement ...
1
vote
1answer
10 views
Is it appropriate to use a confusion matrix for frequency data?
I am testing the accuracy of a machine learning approach that counts cars in images. I have both a predicted dataset and a "real" dataset that was generated by a human. For example, this is what my ...
0
votes
0answers
48 views
Accuracy score or AUC extracted from Gradient Boosting Classifier of scikit-learn? [duplicate]
I'm working on developing a predictive model for a binary classification problem related to biomedical applications (need a really high and promising accuracy). I'm training on my training dataset and ...
5
votes
1answer
95 views
Should OOB (Out Of Bag) error be less than a Test set error in Random Forests?
I am using the book, "An introduction to statistical learning with applications in R" and reading the section on using OOB to estimate the model error for Random Forests. The graph seems to suggest ...
0
votes
0answers
15 views
Acceptable Accuracy, Precision, Sensitivity, and Specificity Thresholds [duplicate]
Are there general rule of thumbs for acceptable accuracy, precision, sensitivity, and specificity values/thresholds in classification? I would imagine that this depends on different applications. I ...
0
votes
1answer
49 views
Mean Absolute Scaled Error implementation on multistep time series forecast
The formula for MASE can be found here: https://en.wikipedia.org/wiki/Mean_absolute_scaled_error
I am building a multi-step time series forecaster and I want to use MASE as a measure of prediction ...
0
votes
0answers
34 views
How can MASE (Mean Absolute Scaled Error) score value be interpreted for non time series data?
If I have used MASE to calculate non time-series data error (as described by Dr. Rob Hyndman here), how can I know if the score received is good or not? Since it is not a time-series, a random walk ...
1
vote
0answers
10 views
What measures of an ML algorithm's 'accuracy' are mostly consistent as the number of classes to predict into varies?
For a research project, I've got a bunch (N=507) of 20-second VR tracking data clips (6DOF x head and hands), each from a different participant. My goal is to predict the participant using a small ...
3
votes
1answer
45 views
Including Collider Variables in Prediction
When the goal is to estimate a causal association between X and Y in the regression framework, one should not condition on (include as covariates) collider variables (common causes of both X and Y) ...
0
votes
1answer
199 views
why the accuracy of my CNN decreasing after some epochs?
at high accuracy, after some epochs the accuracy as well as validation accuracy is decreasing and got stuck after few more epochs. i dont understand why this happened. does more epochs at some point ...
0
votes
0answers
14 views
increasing accuracy of binary logistic regression by reducing type II error?
I am to use binary logistic regression predict a deadly disease from 109 cases out of 385 patients. If during the preliminary diagnosis all patients were sent to the expert doctor for secondary ...
0
votes
1answer
32 views
accuracy and precision in regression vs classification
Are accuracy and precision the same things in regression and classification?
In regression: accuracy is bias, and precision is inverse of variance.
In classification: accuracy is correct prediction ...
0
votes
1answer
42 views
improving accuracy of classification model
I have data with 95 numeric variables and 5 categorical variables. My Y has 2 values. I built a decision tree and my accuracy was 81.8%. Then I created 3 new variables as follows. It improved accuracy ...
0
votes
0answers
30 views
Machine Learning: Model doesn´t recognize letters but has 80% accuracy
I have build a model to classify numbers and characters on Images. I trained it on the Chars74K dataset and in training it has 80% validation accuracy. I just use the number and uppercase characters ...
0
votes
0answers
371 views
Leave One Out Cross-Validation in Python
For me is not clear the way to implement LOOCV in Python, I have the next Python scripts:
1) ACC = 76.92 %
...
0
votes
0answers
21 views
False negative probability of cross-correlation
Suppose $D$ is a string of length $L$ where its values are uniformly distributed in $[-a,a]$. Also, all values of $D$ are independent (i.i.d).
$X$ is a noisy version of $D$ in this way: $X=D+N$ where ...
0
votes
0answers
24 views
Interpreting accuracy values in ARIMA
How to interpet each measuring accuracy and how will i know if its accurate measure in the model
