Skip to main content

Questions tagged [precision]

Precision is about variability while accuracy (in contrast to precision) is about bias. This tag pertains to measurement or estimation; use [precision-recall] when talking about classifiers.

Filter by
Sorted by
Tagged with
4 votes
3 answers
497 views

Judging a model through the TP, TN, FP, and FN values

I am evaluating a model that predicts the existence or not existence of a "characteristic" (for example, "there is a dog in this image") using several datasets. The system outputs ...
KansaiRobot's user avatar
2 votes
1 answer
23 views

Precision calculation for Test data

I have a trained multiclassification (4 different labels) ML model for which I calculated Accuracy and Precision using Confusion Matrix . Now for the developed model, I give some test data without ...
Pavithra 's user avatar
10 votes
2 answers
115 views

What is the proper formatting of very low p values? [closed]

Generally, I believe in giving exact p values rather than something like " p < 0.05". But, sometimes, the p value is extremely low, on the order, say, of $10^{-10}$ or even lower. Under ...
Peter Flom's user avatar
  • 125k
2 votes
1 answer
83 views

We have sensitivity-specificity space (ROC curves) and precision-recall space (PR curves, $F_1$ score). What work has been done with PPV-NPV space?

Receiver-operator characteristic (ROC) curves display the balance between sensitivity and specificity: how good you are at detecting category $1$ (sensitivity) while not falsely identifying category $...
Dave's user avatar
  • 64.7k
1 vote
0 answers
31 views

What is the difference between Confidence intervals and precision intervals? [closed]

I've tried searching but I cannot even find a definition for a "precision interval" anyway. I am completing a meta-analysis and the package in R that I used to create my figure put both ...
maggo's user avatar
  • 11
3 votes
2 answers
330 views

Is F-score the same as accuracy when there are only two classes of equal size?

The title says it all: Is F-score the same as accuracy when there are only two classes of equal sizes? For my specific case, I have measurements of a group of people under two different situations and ...
user1596274's user avatar
3 votes
1 answer
81 views

Classification metrics in regression: can an analogue to precision, for instance, make sense on a continuum?

If we have a true classifier, it can make sense to calculate measures of performance like accuracy, precision (positive predictive value), and recall (sensitivity). Each of these has something to do ...
Dave's user avatar
  • 64.7k
0 votes
0 answers
16 views

Precision-based Sample Size Calculation for Repeated Measures Design

I'm looking to calculate the needed sample size for a study based on precision of estimates, but am having a very difficult time figuring out exactly how to do it. What is mainly throwing me off is ...
Jake Remmert's user avatar
4 votes
2 answers
71 views

Is there a standard measurement of 2-dimensional precision?

Suppose a gun is fired at the bullseye of a target a specific number of times. Ignoring the accuracy (e.g. all the shots are well to the left and above the bullseye), is there a standard way of ...
Ray Butterworth's user avatar
3 votes
2 answers
163 views

Calculate area under precision-recall curve from area under ROC curve and the prevalence

I am reading material that reports the area under a ROC curve. I am curious to know what the performance would be in precision-recall space. From the sensitivity and specificity values in the ROC ...
Dave's user avatar
  • 64.7k
1 vote
0 answers
46 views

Confidence intervals for Object Detection metrics

I would like to come back on this "When do we require to calculate the confidence Interval?" since recently a reviewer asked me to provide confidence intervals for metrics regarding my work ...
rok's user avatar
  • 111
6 votes
3 answers
2k views

Is it okay to say that 95% confidence interval is more significant than 80%?

So, the higher the confidence interval the lower the false positive rate, but the false negative rate will increase lowering the recall. Is it possible to determine which confidence interval is better/...
Ankita's user avatar
  • 129
1 vote
0 answers
109 views

Explanation needed for different ways of calculating precision (ISO-5725 and CLSI EP05-A2)

I'm trying to get along with calculating precision of analytical method. I'm a chemist not a mathematician. ICH M10 guideline requires that: Accuracy and precision should be determined by analysing ...
Radek Jaźwiec's user avatar
8 votes
3 answers
698 views

How does population size impact the precision of the results?

Suppose we want to conduct a poll among a sample from the population. I know that increasing a sample size generally improves the precision of the results, but I was wondering whether given fixed ...
RabbitTender's user avatar
3 votes
1 answer
41 views

Precision and recall reported in classification model

I have one question about the evaluation metrics of classification models. I see many people report the precision and recall value for their classification models. Do they choose a threshold to ...
Salty Gold Fish's user avatar
1 vote
1 answer
48 views

combine specificity and

I am performing classification on an imbalanced dataset (70% negatives). If a prediction is negative I take a specific action otherwise an opposite one. As in both cases some costs are implied, I want ...
shamalaia's user avatar
  • 285
4 votes
2 answers
547 views

ROC/AUC Curve for False Negatives (Type 2 Errors)?

I keep seeing this curve and I understand this basically tells you how well the model is doing in terms of predicting True positives vs. False positives. I was wondering if there is a version of this ...
Katsu's user avatar
  • 1,011
2 votes
1 answer
32 views

How to determine the precision of a measurement using a "perfect" reference method?

I developed an experimental method that measures certain material property. I want to see how precise my method is, so I have a confidence interval, or other metric. To do this, I'm comparing it with ...
user46147's user avatar
  • 135
2 votes
1 answer
1k views

Average Precision (AP) for object detection, huge confusion

I've been reading about how object detection models are evaluated. It seems that the metric most often used is AP. But I have stumbled upon 2 different approaches that I think mean completely ...
Tomé Silva's user avatar
5 votes
4 answers
3k views

Precision vs. specificity

I know that if we cannot afford to have false positive results, we should aim for high precision. My question is, how is precision different from specificity? Any examples?
MxML's user avatar
  • 71
1 vote
2 answers
465 views

Combine accuracy, precision, and recall

I am working on a classification problem. Several models are produced and all have accuracy, precision and recall metrics on test data. I need to pick the best model among the alternatives. What I can ...
Emin Ozkan's user avatar
2 votes
1 answer
26 views

Basic questions about the best way to report precision in quantitative assay validation study

This feels like a very basic question, but I'm feeling stuck on it. I'm writing up a biological assay development/validation study for a method of quantitating multiple organisms from a mixture ...
MikeyC's user avatar
  • 121
1 vote
0 answers
73 views

How to account for overlap in individual studies in meta-analysis of meta-analyses (umbrella analysis)

I want to conduct a meta-analysis of meta-analyses (aka umbrella analysis). A potential issue is that within each individual meta-analysis I want to include there is some overlap between the studies. ...
Ajj1988's user avatar
  • 51
1 vote
0 answers
552 views

Noninformative prior for Gaussian precision as the special case of Gamma distribution with both parameters being zero

The question is from page 120 of book "Pattern Recognition and Machine Learning" by Christopher M. Bishop. I excerpt it as follows: The definition of gamma distribution $\textrm{Gam}(\...
zzzhhh's user avatar
  • 333
0 votes
0 answers
28 views

Understanding order of operations in formula

Can someone help me understand the order of operations for this formula? Lets say: y_estimated = 10, 14, 11 #three different estimates that will be subbed into the formula y_true= 12 R=3 I think it ...
hugh_man's user avatar
  • 101
1 vote
1 answer
963 views

Does it matter if real data will be imbalanced, if the ML model was trained on a balanced dataset?

I have trained a machine learning model (supervised, classification, LinearSVC) on a balanced dataset, which produces relatively good results on the test data. I am happy with the numbers, but not ...
zuccinni's user avatar
0 votes
1 answer
234 views

Can I conclude that the classifier is always good when Precision-Recall Curve above the baseline?

I used logistic regression for highly imbalaned data (1=0.6% , 0=99.4%) Since PR curves are sensitive to imbalance, so i used it, but I don't know how to interpret graph appropriately. This is PR-...
Rhee Eunhee's user avatar
0 votes
0 answers
41 views

A question about precision

We have 2 Classification models (Random forest on balanced Data Set), the first one classify a Bank's client as a Churner (closing his account) or an active client, the second one classify a Bank's ...
yassine ben's user avatar
0 votes
0 answers
266 views

High precision and low recall but with a balanced dataset

When i evaluated my model (CatBoost classifier), I noticed that my model has high precision and low recall (Recall: 0.59, Precision: 0.89) but the classes are perfectly balanced (1: 45.5, 2: 54.5) and ...
Federico Mele's user avatar
0 votes
0 answers
31 views

Determining the precision of a range

Supposing that I have 2 measured values which cover a range Y and Z and the actual value "X" is somewhere in the middle. How can I calculate the precision of the range? I expected to ...
CipherJunk's user avatar
1 vote
1 answer
65 views

Sensibly dealing with the precision available in likelihood functions

I am running some simulations involving Bayesian updating of prior odds given a succession of measurements and corresponding likelihood functions. Inevitably, repeated multiplication of the prior by ...
CrimsonDark's user avatar
1 vote
1 answer
474 views

classification ML model: probability of positive label knowing the model score

Question at the intersection of ML and statistics. I built a binary classification ML model, that for each input observation x outputs the probability p(x) in (0,1) that x belongs to the positive ...
Niccolo''s user avatar
  • 121
1 vote
2 answers
2k views

Are evaluation metrics computed on training dataset?

Based on my own studies and questions on this site, my understanding is that evaluation metrics (accuracy, precision, recall) are only calculated on the test dataset. The training dataset is used to ...
miniMint's user avatar
  • 113
0 votes
1 answer
113 views

How can precision be less than one in Leave-One-Subject-Out binary classification if each subject contains only one class

Say I'm trying to classify a medical condition. Theres only two classes: Sick and Healthy. I build a model and I can't split the data because I don't want data from the same patient being in training ...
IsmailE's user avatar
2 votes
0 answers
39 views

Meta-analysis on percentage of total calorie intake

We are looking at a number of studies which have profiled nutrient information which contributes to total daily calorie intake. For example, fibre might make up 5% of daily total calorie intake. My ...
Ajj1988's user avatar
  • 51
3 votes
1 answer
3k views

Is bias and variance equivalent to accuracy vs precision?

I looked at some graphs of high bias and high variance, and I don't really understand how the bias vs variance tradeoff is any different from accuracy vs precision?
Fine-Tuning's user avatar
1 vote
1 answer
110 views

Inaccurate parameter estimates for state-space models?

I'm modeling time series using dynamic linear/state-space models, and was surprised by how inaccurate the estimates of model parameters can be, even for fairly long series. In particular, I'm ...
Mark Wexler's user avatar
0 votes
1 answer
1k views

What is the average precision in the case of no positives for a given category in the context of object detection

In attempting to calculate the average precision of an object detection model, I am wondering about an edge case. Suppose at evaluation time that for a given category, that no detections of that ...
IntegrateThis's user avatar
8 votes
1 answer
8k views

How many significant figures should I report for a regression equation?

I have a regression equation that I am trying to include as part of a study predicting a variable of interest using a linear proxy using a simple ordinary least squares function. I did the regression ...
user2352714's user avatar
0 votes
0 answers
23 views

Precision-recall curve for two class problem

I'm wondering how to compute the precision-recall curve for the following problem: Let's say the true values are [1 0 1 1 1] Predicted values are [1 1 0 1 1] We have two values only that were not ...
Badrawi's user avatar
3 votes
1 answer
905 views

Why adjusting for randomization stratification factors in the model improves the precision of estimators for treatment effect?

I assume you will already get balanced treatment assignment within the randomization strata. Why would we still gain improvement in precision by adjusting for these factors in the model? Does not ...
hehe's user avatar
  • 743
2 votes
0 answers
749 views

Precision-Recall Curve Intuition for Multi-Class Classification

I am running a CNN image multi-class classification model with Keras/Tensorflow and have established about a 90% overall accuracy with my best model trial. I have 10 unique classes I am trying to ...
Coldchain9's user avatar
1 vote
2 answers
428 views

Question about OLS and BLUE in the presence of hetereoscadasticity and robust standard errors

My understanding that if errors are non-spherical, OLS is no longer the minimum variance linear unbiased estimator (assume the error terms are fully independent of all covariaties- so unbiasedness ...
Steve's user avatar
  • 681
1 vote
1 answer
580 views

Event Study regression standard errors

When running an 'event study' diff in diff, i.e. : $y_{i,t} = \sum_{k\neq-1}\beta_k *1\{t=k\} + \lambda_t + \mu_i +error$ where i is a group level(i.e. individual, county etc). and t is time, $\...
Steve's user avatar
  • 681
5 votes
2 answers
1k views

Disadvantage of precision at k

Suppose 10 documents were retrieved (rectangle with black color is relevant document). In the following table, Precision @ k is calculated. P@10 or "Precision at 10" corresponds to the ...
GoDev's user avatar
  • 195
3 votes
1 answer
3k views

False positive rate at K recall

I just stumbled upon a new metric I've never heard about called False Positive rate at K recall (FPR-K). Searching the internet I just managed to find more papers using the metric but none of them ...
Jjang's user avatar
  • 241
1 vote
1 answer
2k views

Should I balance the classifier train/test set, if metrics is Precision/Recall (F1 score)?

I want to train a classifier on an unbalanced data set. Proportions of classes C0/C1 are 65/35. Importantly, the success metrics is F1_score. In other words, the proper classification of class 1 (...
Data Man's user avatar
1 vote
1 answer
154 views

Obtaining Raw Probabilities From OVR Logistic Regression

I am relatively new to machine learning and I am working on a multi-class classification problem. I am attempting to utilize OVR logistic regression. When you run through an OVR model, the end result ...
Kayla N's user avatar
  • 11
4 votes
2 answers
1k views

Should I calculate the mean and standard deviation with raw or transformed data?

I'm an undergrad chemistry student, and in a recent laboratory session, we were given a set of observations for the volume of a solution in order to find an unknown concentration of a reactant $R$, ...
user avatar
0 votes
1 answer
185 views

Including measurement precision in a bayesian linear model

I'm using Jags to fit a Bayesian linear regression to a dataset. The model is: N[i]∼N(μ[i],τ) with precision τ and mean: ...
Sovay's user avatar
  • 11