Questions tagged [accuracy]
Accuracy of an estimator is the degree of closeness of the estimates to the true value. For a classifier, accuracy is the proportion of correct classifications. (This second usage is not good practice. See the tag wiki for a link to further information.)
0
votes
0answers
5 views
0
votes
0answers
9 views
2
votes
1answer
30 views
Which approach should be used to compare two different measurement techniques of same samples?
I have individually measured failure forces of 8 materials and those recorded with A method and B method in same time: 8 results in each method, A=8 and B=8. The range of data of both measurement ...
1
vote
1answer
7 views
Is it appropriate to use a confusion matrix for frequency data?
I am testing the accuracy of a machine learning approach that counts cars in images. I have both a predicted dataset and a "real" dataset that was generated by a human. For example, this is what my ...
0
votes
0answers
25 views
Accuracy score or AUC extracted from Gradient Boosting Classifier of scikit-learn? [duplicate]
I'm working on developing a predictive model for a binary classification problem related to biomedical applications (need a really high and promising accuracy). I'm training on my training dataset and ...
0
votes
0answers
4 views
how to find a accuracy in weka for clustering? [closed]
Can anyone please tell me how to find a accuracy for k-means clustering in weka?
5
votes
1answer
45 views
Should OOB (Out Of Bag) error be less than a Test set error in Random Forests?
I am using the book, "An introduction to statistical learning with applications in R" and reading the section on using OOB to estimate the model error for Random Forests. The graph seems to suggest ...
0
votes
0answers
15 views
Acceptable Accuracy, Precision, Sensitivity, and Specificity Thresholds [duplicate]
Are there general rule of thumbs for acceptable accuracy, precision, sensitivity, and specificity values/thresholds in classification? I would imagine that this depends on different applications. I ...
0
votes
0answers
8 views
Mean Absolute Scaled Error implementation on multistep time series forecast
The formula for MASE can be found here: https://en.wikipedia.org/wiki/Mean_absolute_scaled_error
I am building a multi-step time series forecaster and I want to use MASE as a measure of prediction ...
0
votes
0answers
12 views
How can MASE (Mean Absolute Scaled Error) score value be interpreted for non time series data?
If I have used MASE to calculate non time-series data error (as described by Dr. Rob Hyndman here), how can I know if the score received is good or not? Since it is not a time-series, a random walk ...
1
vote
0answers
10 views
What measures of an ML algorithm's 'accuracy' are mostly consistent as the number of classes to predict into varies?
For a research project, I've got a bunch (N=507) of 20-second VR tracking data clips (6DOF x head and hands), each from a different participant. My goal is to predict the participant using a small ...
2
votes
1answer
39 views
Including Collider Variables in Prediction
When the goal is to estimate a causal association between X and Y in the regression framework, one should not condition on (include as covariates) collider variables (common causes of both X and Y) ...
0
votes
1answer
56 views
why the accuracy of my CNN decreasing after some epochs?
at high accuracy, after some epochs the accuracy as well as validation accuracy is decreasing and got stuck after few more epochs. i dont understand why this happened. does more epochs at some point ...
0
votes
0answers
12 views
increasing accuracy of binary logistic regression by reducing type II error?
I am to use binary logistic regression predict a deadly disease from 109 cases out of 385 patients. If during the preliminary diagnosis all patients were sent to the expert doctor for secondary ...
0
votes
1answer
22 views
accuracy and precision in regression vs classification
Are accuracy and precision the same things in regression and classification?
In regression: accuracy is bias, and precision is inverse of variance.
In classification: accuracy is correct prediction ...
0
votes
1answer
29 views
improving accuracy of classification model
I have data with 95 numeric variables and 5 categorical variables. My Y has 2 values. I built a decision tree and my accuracy was 81.8%. Then I created 3 new variables as follows. It improved accuracy ...
0
votes
0answers
27 views
Machine Learning: Model doesn´t recognize letters but has 80% accuracy
I have build a model to classify numbers and characters on Images. I trained it on the Chars74K dataset and in training it has 80% validation accuracy. I just use the number and uppercase characters ...
0
votes
0answers
40 views
Leave One Out Cross-Validation in Python
For me is not clear the way to implement LOOCV in Python, I have the next Python scripts:
1) ACC = 76.92 %
...
0
votes
0answers
19 views
False negative probability of cross-correlation
Suppose $D$ is a string of length $L$ where its values are uniformly distributed in $[-a,a]$. Also, all values of $D$ are independent (i.i.d).
$X$ is a noisy version of $D$ in this way: $X=D+N$ where ...
0
votes
0answers
12 views
Interpreting accuracy values in ARIMA
How to interpet each measuring accuracy and how will i know if its accurate measure in the model
0
votes
1answer
87 views
What is difference between accuracy_score() and cross_val_score()?
The problem I'm working on is a multiclass-classification. Have been reading through lot of articles and documentation, but not able to figure out which of Accuracy_Score or Cross_Val_Score should be ...
0
votes
0answers
8 views
Is it possible to have a constent accuracy for my tensorflow network whatever is the iterations?
I am runing my tf NN composed by one input, one hidden and one output layers the number of hidden nodes if the half of total features number I have got the next table considering different learning ...
0
votes
1answer
48 views
A standard name for a formula to “Maximize true positives while minimize false positives”
I am using an evaluation metric to reward the true positives and penalize the false positive ones retrieved by a function $f(\cdot)$.
Indeed, it can be represented as follows:
$\frac{\texttt{|TP|} - \...
0
votes
0answers
9 views
Which performance metric to use for stratified data? [duplicate]
I'm trying to classify a data into 3 classes (supervised), one of which is heavily underrepresented in the data set. In order to combat this imbalance, I decided to stratify the data. Now I want to ...
0
votes
1answer
43 views
Speeding up in R [closed]
for(i in 1 : n){
y[i] <- length(X[X >= X[i]])
}
This is my code in R (partially given) to obtain the number of X's greater than or equal to each Xi, where X is a vector of values. When I run ...
1
vote
1answer
130 views
What is the baseline of the F1 score for a binary classifier?
I know how to calculate the baseline for the accuracy of a binary classification problem: I simply always predict the majority class, e.g. if there is 94% True values and 6% False values, my baseline ...
0
votes
0answers
33 views
Disproportionate classification accurary between testset and entire dataset (Random Forest)
So I have a multiclass problem (16 classes, 58k samples), for which I decided to use the RandomForestClassifier.
After some feature engineering and cross-validation I got a test set (13k samples) ...
0
votes
0answers
11 views
When do we use precision/recall over accuracy when evaluating a classifier? [duplicate]
When do we use precision/recall over accuracy when evaluating a classifier? What kind of scenarios would mean precision/recall metrics would be better than accuracy?
0
votes
0answers
13 views
Why does fully convolutional network plateau first and then learns?
Im training a fully convolutional network to classify handwriting Chinese characters. The dev dataset I am using has 250 classes with 200 - 300 samples in each class.
And I found out no matter how I ...
0
votes
1answer
26 views
Can I compute an F1 score when the test data has no examples of one class?
I am working on a 3-class classification problem. We are cross-validating via a Leave-One-Out Approach, and there are some instances where the test data has no instances of one of my three classes. ...
0
votes
0answers
19 views
Regarding drop in test accuracy of one fold in 5 fold cross validation
I am training and testing a convolution neural network for binomial classification with 5 fold cross-validation. The data set has 4684 3 dimensional arrays of shape (2000,102,1). This is a sparse data ...
2
votes
1answer
32 views
Best strategy to maximize the prediction accuracy when p >> n
I am solving the following classification problem:
thousands of features, but only 40 samples (i.e. p >> n)
classes are balanced
it is not possible to get more data
the only thing I am interested in ...
0
votes
0answers
21 views
Why isn't accuracy of binary classification model improving? [duplicate]
I have a data set with a binary response variable, about 30,000 observations of 8 features, some are continuous and some are categorical.
This is an imbalanced data set, the ratio of negatives to ...
0
votes
0answers
31 views
Confidence Interval for accuracy - Binomial?
Imagine we fit the same data mining model on 100 different validation datasets and accuracy is always exactly 95%. If we now use the Binomial proportion confidence interval we would have a standard ...
1
vote
0answers
40 views
Measure of accuracy for a Bayesian model
I am reading Statistical Rethinking (Section 6.2.1.2).
The topic of this section is measuring accuracy for a Bayesian model, i.e. accuracy of the model of predicting correctly an outcome.
The ...
1
vote
1answer
35 views
With test accuracy being equal, is it better to have lower training accuracy?
Suppose we train two models on a training set, and then test them both on the training set itself, and on a test set. We have some accuracy metric we're using to evaluate them.
Both models score ...
0
votes
1answer
38 views
What is the reasons for a model to have a high cross validation score and yet underperforms on unseen data?
I have a model that is based on an experiment collected on 100 subjects. We are testing the model as follows:
Record raw data from the subjects
For each subject, compute the feature from the raw data ...
0
votes
0answers
23 views
What is the efficient algorithm among Bagged Trees, Neural Network (TensorFlow) and C4.5?
I am performing a classification process over a collection of signals where each signal has 12 parameter. I need to predict my class/ label using those 12 features, the classes are 5 from 0 to 4 (It ...
1
vote
0answers
47 views
Naive Bayes and kNN accuracy
Assume that a large number of binary features are added to a dataset with two
class labels c1 and c2, such that for each added feature f, the class conditional
probability P(f = 0|c1) = P(f = 0|c2). ...
3
votes
1answer
122 views
why too many epochs will cause overfitting?
I am reading the 《deep learning with python》.
In chapter 4, about Fighting overfitting, I have two questions.
why increasing epochs may cause overfitting?
I know ...
1
vote
2answers
231 views
What is the name for the complement of accuracy?
I have a metric that is defined as $1 - Accuracy$ and I need a name for it. Is there a scientific name for the complement of accuracy?
0
votes
0answers
33 views
Test Set Probabilities and Accuracy [duplicate]
Say we've got a logistic regression model $M$ used as a classifier in a binary case. Now we take a test set $\tau=\{(x_1,y_1),...,(x_n,y_n)\}$, each test sample is assigned with $\hat{\pi}_i=P(y_i=1|...
1
vote
0answers
30 views
How to know if your decision tree model has overfitting or not?
I am using the DecisionTreeClassifier() of python and I am changing some tuning parameters to understand if my model has overfitting or no because when I exectue the following code without any ...
0
votes
0answers
24 views
What does the average accuracy with decay mean when training the model?
I saw this formula in a code in github of a paper and I did not get its meaning.
avg_accuracy += (1-accuracy_decay) * (accuracy-avg_accuracy)
And they ...
1
vote
1answer
17 views
Evaluate classifier based on predicted probabilities?
If I had a categorical response $Y$ and multiple categorical features $X$, and I wanted to fit a model to predict $Y$.
If all I cared about was the eventual distribution of $Y$ (say in terms of %), I ...
1
vote
0answers
47 views
How to correctly compare the accuracy of different forecasting methods using bootstrapping with time series forecasting
I am currently working on a forecasting project and I have tried several different models to forecast with. Having trained and tuned my models I want to pick which model works best for each time ...
0
votes
1answer
16 views
Polluting a dataset with “non-determinables”
Let's say I'm working on solving sudoku puzzles with machine learning. Now, plenty of good methods exist for solving sudoku algorithmically, no machine learning required, but let's play along to get ...
0
votes
0answers
59 views
Can ICC be used to measure and benchmark ordinal classification accuracy?
I am conducting ordinal regression with an ordinal categorical target outcome roughly equally distributed as 1, 2 or 3. (This ordinal categorical target is actually the conversion of a numeric target ...
1
vote
0answers
27 views
Boosted decision trees: in which situations are “deep” decision trees performing better?
The general idea of boosted decision trees is to use very simple trees in the following manner (simplified, for intuition only):
start with a simple tree,
fit another simple tree on the residuals,
...
0
votes
1answer
81 views
Checking the similarity of two classes in a binary classification
So, I have a binary classification problem. The classes are fairly balanced and I have a separate training set and a test set. No matter what I try, both classification accuracy and the f1 score are ...
