Stack Exchange Network

Stack Exchange network consists of 174 Q&A communities including Stack Overflow, the largest, most trusted online community for developers to learn, share their knowledge, and build their careers.

Visit Stack Exchange

Questions tagged [accuracy]

Accuracy of an estimator is the degree of closeness of the estimates to the true value. For a classifier, accuracy is the proportion of correct classifications. (This second usage is not good practice. See the tag wiki for a link to further information.)

0
votes
0answers
8 views

Is it possible to have a constent accuracy for my tensorflow network whatever is the iterations?

I am runing my tf NN composed by one input, one hidden and one output layers the number of hidden nodes if the half of total features number I have got the next table considering different learning ...
0
votes
1answer
26 views

A standard name for a formula to “Maximize true positives while minimize false positives”

I am using an evaluation metric to reward the true positives and penalize the false positive ones retrieved by a function $f(\cdot)$. Indeed, it can be represented as follows: $\frac{\texttt{|TP|} - \...
0
votes
0answers
8 views

Which performance metric to use for stratified data? [duplicate]

I'm trying to classify a data into 3 classes (supervised), one of which is heavily underrepresented in the data set. In order to combat this imbalance, I decided to stratify the data. Now I want to ...
0
votes
1answer
42 views

Speeding up in R [closed]

for(i in 1 : n){ y[i] <- length(X[X >= X[i]]) } This is my code in R (partially given) to obtain the number of X's greater than or equal to each Xi, where X is a vector of values. When I run ...
1
vote
1answer
29 views

What is the baseline of the F1 score for a binary classifier?

I know how to calculate the baseline for the accuracy of a binary classification problem: I simply always predict the majority class, e.g. if there is 94% True values and 6% False values, my baseline ...
0
votes
0answers
23 views

Disproportionate classification accurary between testset and entire dataset (Random Forest)

So I have a multiclass problem (16 classes, 58k samples), for which I decided to use the RandomForestClassifier. After some feature engineering and cross-validation I got a test set (13k samples) ...
0
votes
0answers
10 views

When do we use precision/recall over accuracy when evaluating a classifier? [duplicate]

When do we use precision/recall over accuracy when evaluating a classifier? What kind of scenarios would mean precision/recall metrics would be better than accuracy?
0
votes
0answers
10 views

Why does fully convolutional network plateau first and then learns?

Im training a fully convolutional network to classify handwriting Chinese characters. The dev dataset I am using has 250 classes with 200 - 300 samples in each class. And I found out no matter how I ...
0
votes
1answer
16 views

Can I compute an F1 score when the test data has no examples of one class?

I am working on a 3-class classification problem. We are cross-validating via a Leave-One-Out Approach, and there are some instances where the test data has no instances of one of my three classes. ...
0
votes
0answers
14 views

Regarding drop in test accuracy of one fold in 5 fold cross validation

I am training and testing a convolution neural network for binomial classification with 5 fold cross-validation. The data set has 4684 3 dimensional arrays of shape (2000,102,1). This is a sparse data ...
2
votes
1answer
25 views

Best strategy to maximize the prediction accuracy when p >> n

I am solving the following classification problem: thousands of features, but only 40 samples (i.e. p >> n) classes are balanced it is not possible to get more data the only thing I am interested in ...
0
votes
0answers
19 views

Why isn't accuracy of binary classification model improving? [duplicate]

I have a data set with a binary response variable, about 30,000 observations of 8 features, some are continuous and some are categorical. This is an imbalanced data set, the ratio of negatives to ...
0
votes
0answers
20 views

Confidence Interval for accuracy - Binomial?

Imagine we fit the same data mining model on 100 different validation datasets and accuracy is always exactly 95%. If we now use the Binomial proportion confidence interval we would have a standard ...
1
vote
0answers
29 views

Measure of accuracy for a Bayesian model

I am reading Statistical Rethinking (Section 6.2.1.2). The topic of this section is measuring accuracy for a Bayesian model, i.e. accuracy of the model of predicting correctly an outcome. The ...
1
vote
1answer
34 views

With test accuracy being equal, is it better to have lower training accuracy?

Suppose we train two models on a training set, and then test them both on the training set itself, and on a test set. We have some accuracy metric we're using to evaluate them. Both models score ...
0
votes
1answer
36 views

What is the reasons for a model to have a high cross validation score and yet underperforms on unseen data?

I have a model that is based on an experiment collected on 100 subjects. We are testing the model as follows: Record raw data from the subjects For each subject, compute the feature from the raw data ...
0
votes
0answers
20 views

What is the efficient algorithm among Bagged Trees, Neural Network (TensorFlow) and C4.5?

I am performing a classification process over a collection of signals where each signal has 12 parameter. I need to predict my class/ label using those 12 features, the classes are 5 from 0 to 4 (It ...
1
vote
0answers
24 views

Naive Bayes and kNN accuracy

Assume that a large number of binary features are added to a dataset with two class labels c1 and c2, such that for each added feature f, the class conditional probability P(f = 0|c1) = P(f = 0|c2). ...
3
votes
1answer
43 views

why too many epochs will cause overfitting?

I am reading the 《deep learning with python》. In chapter 4, about Fighting overfitting, I have two questions. why increasing epochs may cause overfitting? I know ...
1
vote
2answers
225 views

What is the name for the complement of accuracy?

I have a metric that is defined as $1 - Accuracy$ and I need a name for it. Is there a scientific name for the complement of accuracy?
0
votes
0answers
33 views

Test Set Probabilities and Accuracy [duplicate]

Say we've got a logistic regression model $M$ used as a classifier in a binary case. Now we take a test set $\tau=\{(x_1,y_1),...,(x_n,y_n)\}$, each test sample is assigned with $\hat{\pi}_i=P(y_i=1|...
1
vote
0answers
29 views

How to know if your decision tree model has overfitting or not?

I am using the DecisionTreeClassifier() of python and I am changing some tuning parameters to understand if my model has overfitting or no because when I exectue the following code without any ...
0
votes
0answers
14 views

What does the average accuracy with decay mean when training the model?

I saw this formula in a code in github of a paper and I did not get its meaning. avg_accuracy += (1-accuracy_decay) * (accuracy-avg_accuracy) And they ...
1
vote
1answer
16 views

Evaluate classifier based on predicted probabilities?

If I had a categorical response $Y$ and multiple categorical features $X$, and I wanted to fit a model to predict $Y$. If all I cared about was the eventual distribution of $Y$ (say in terms of %), I ...
1
vote
0answers
37 views

How to correctly compare the accuracy of different forecasting methods using bootstrapping with time series forecasting

I am currently working on a forecasting project and I have tried several different models to forecast with. Having trained and tuned my models I want to pick which model works best for each time ...
0
votes
1answer
16 views

Polluting a dataset with “non-determinables”

Let's say I'm working on solving sudoku puzzles with machine learning. Now, plenty of good methods exist for solving sudoku algorithmically, no machine learning required, but let's play along to get ...
0
votes
0answers
26 views

Can ICC be used to measure and benchmark ordinal classification accuracy?

I am conducting ordinal regression with an ordinal categorical target outcome roughly equally distributed as 1, 2 or 3. (This ordinal categorical target is actually the conversion of a numeric target ...
1
vote
0answers
26 views

Boosted decision trees: in which situations are “deep” decision trees performing better?

The general idea of boosted decision trees is to use very simple trees in the following manner (simplified, for intuition only): start with a simple tree, fit another simple tree on the residuals, ...
0
votes
1answer
55 views

Checking the similarity of two classes in a binary classification

So, I have a binary classification problem. The classes are fairly balanced and I have a separate training set and a test set. No matter what I try, both classification accuracy and the f1 score are ...
2
votes
1answer
72 views

How to meaningfully compute the accuracy of a multi-step forecast produced by a model

I am trying to measure the accuracy of my model in producing a multi-step forecast and I have read a lot of different opinions on the matter and am now rather confused. The goal of my model is to ...
0
votes
2answers
34 views

Can I create a test dataset with known errors to validate accuracy assessment? [closed]

I developed a procedure to measure the geometric accuracy of 3D building models based on the similarity to a 3D point cloud. Therefore I created mainly two quality criteria. The result of my automatic ...
1
vote
0answers
24 views

Compare accuracy between tools using k-fold cross validation, each tool is tested with different k values

I'm working on a new way to do the classification in a supervised way and I want to compare its accuracy to some related works. These works are using the same data set and they are testing their ...
1
vote
0answers
128 views

Calculation of accuracy (and Cohen's kappa) using sensitivity, specificity, positive and negative predictive values

I read How to calculate specificity from accuracy and sensitivity, but I have two diagnostic performance measures more. Please correct me if I am wrong: if Sensitivity=TP/(TP+FN) Specificity=TN/(TN+...
1
vote
1answer
21 views

Quality metric for classifier with decision rule allowing “none of the above”

Let's suppose that I have classification model for n classes ($n>1$). The classifier returns a probability distribution over a set of classes. But if classifier is not sure (i.e. there is no ...
0
votes
0answers
19 views

Fuzzy String Matching and Word Length

I'm attempting to calculate how likely it is that two words are the "same", but misspelled. I've narrowed it down to a few algorithms that will give me a similarity metric, but the problem is that ...
1
vote
0answers
53 views

Confusions about Pesaran & Timmermann test (2009 version)

I am learning about Pesaran Timmermann test (2009) for directional accuracy and I have troubles understanding its formula. Here I use notation and arrangement used in Pönkä (2017): $$PT=(T-1)(S^{-1}_{...
0
votes
1answer
40 views

Unbalanced dataset accuracy [duplicate]

I’m currently encountering some problems analyzing a dataset with neural network. The problem is that I have an unbalanced binary class training set (10:1). Training accuracy for both classes are 100%....
0
votes
0answers
45 views

AUC and accuracy interpretation

An accuracy $80\%$ of a model that predicts binary outcomes is interpreted as: Given a sample whose outcomes we want to predict, 80% of the prediction will be correct. What does an AUC of $80\%$...
2
votes
1answer
49 views

Constant Accuracy with decreasing loss

I am fairly new to Cross Validated section so I apologize if my question structure is incorrect. I am currently working on Fully Convolutional Networks for Semantic Segmentation. I am first trying ...
0
votes
1answer
70 views

reconstruct a 2X2 confusion matrix (TP, TN, FP, FN) from Sensitivity and Specificity

Is it possible to reconstruct a 2X2 confusion matrix (TP, TN, FP, FN) from Sensitivity, Specificity, Positive Predictive Value, and Negative Predictive Values. I also have prevalence according to the ...
2
votes
1answer
89 views

Classification model accuracy, roc auc score, f1 score 100%

I am working on a binary classification problem. I have split the train set and when I evaluate the model on the validation set all metrics are 100% which is unrealistic considering that I haven't ...
2
votes
1answer
220 views

Example when using accuracy as an outcome measure will lead to a wrong conclusion

I am looking into various performance measures for predictive models. A lot was written about problems of using accuracy, instead of something more continuous to evaluate model performance. Frank ...
1
vote
0answers
32 views

How to make really bad results from a machine learning model better by reversing predictions

I trained a classification model on some data with two classes and have really low accuracy. I have a false-positive rate of 86 % for both classes I am trying to predict. I was wondering if I could ...
0
votes
3answers
106 views

accuracy of a binary classifier vs probability of binomial distribution

I got confused over a simple concept. Imagine that I have a binary classifier with 50% accuracy. So, if there are 10 samples to be classified as "y", "n", it has predicted 5 of them correctly. Now, ...
0
votes
0answers
18 views

Evaluating accuracy of the classifier based on sample?

I have a rather odd question which unfortunately goes beyond my knowledge of stats so any advice will be much appreciated. We built a clustering model on the text data (LDA) and then assigned classes ...
1
vote
1answer
881 views

Difference between Mean/average accuracy and Overall accuracy

I just got confusion while reading the paper "Local Binary Pattern-Based Hyperspectral Image Classification With Superpixel Guidance". They mentioned that they repeated each experiment 10 times and ...
1
vote
1answer
72 views

Does high training accuracy for an NN mean that it has a potential to reach high validation accuracy?

I saw quite a few discussions related to the problem of high training accuracy with low validation accuracy and what steps to take to address it. I have the same problem with a binary classification ...
3
votes
1answer
41 views

Quantitative evaluations for image classification

Hello I am working on the classification of different weed categories. I want to know what quantitative evaluation I can do other than find the accuracy?
0
votes
1answer
37 views

why does data external from training and testing my neural network perform much worse than statistical accuracy?

I've got a problem with my neural network (used to recognize audio signals, an expansion of the UrbanSound dataset problem): when I fit the model the accuracy of both train and validation is near 90%. ...
2
votes
0answers
94 views

Can F1-Score be higher than accuracy?

I'm using sklearn's confusion_matrix and classification_report methods to compute the ...