Skip to main content

Questions tagged [concordance]

Concordance measures the level of agreement when different raters or judges are evaluating the same units. Many different measures of concordance exist, some of them are Krippendortff's alpha and Cohen's kappa. ICC intraclass correlation coefficients are closely related.

Filter by
Sorted by
Tagged with
3 votes
1 answer
46 views

Is it ok to restrict data to uncensored patient in order to compare models

I have read this answer already "removing-censored-observation". I understand that removing censored data induce a bias in the analysis. But I have somehow a specific case. I am just ...
Tamsina Ludwig's user avatar
2 votes
0 answers
40 views

Exact sampling distribution of Kendall's tau under independence (no ties case)

I am looking for an R package for calculating the CDF and the inverse-CDF of the sample Kendall's tau (*) under the null assumption of independence, in the case of no ties. (*) Just to be clear, I am ...
Vicent's user avatar
  • 789
0 votes
0 answers
33 views

Can the Brier score and Concordance index be anti-correlated?

I am using a proportional hazards Cox model to predict the survival probability of some mechanical components. I am using a combined L1-L2 penalization, and I want to optimize the (integrated) Brier ...
Fernando Cucchietti's user avatar
0 votes
0 answers
21 views

How to test similarity between multiple medical differential diagnoses?

I have multiple observations collected by doctors where they give the differential diagnoses, the 10 most probable diseases (d). I want to calculate the inter-observer similarity. my data looks like ...
Timothy's user avatar
0 votes
0 answers
41 views

C-statistic (or AUC) for fractional logistic regression (i.e. continuous regression)

I have proportional data to which I have fit a logistic regression (i.e. fractional logistic regression). The statistician in our group wants me to provide a c-statistic for the regression. My ...
Brandon's user avatar
  • 356
1 vote
0 answers
36 views

Confused on Cohen's Kappa - Exploring Rater Agreement Stats

I am interested in conducting a survey where I show raters a graph then ask for their thoughts. I will do this by providing statement for which the raters will select a value on a 5 point likert scale ...
user21584544's user avatar
1 vote
1 answer
137 views

How to calculate the p-value for the change in C-statistic using boot() or boot.ci()?

I am using boot() and boot.ci() to calculate confidence intervals for the difference in the c -statistic (AUC) between models with and without a novel biomarker. But I don't know how to calculate the ...
shuishui shanshan's user avatar
2 votes
0 answers
35 views

Concordance of an ordinal time series

I have panel data with an ordinal outcome that was assessed by two independent reviewers - i.e. each is unaware of the others' assessment. The time points for assessment are not necessarily concurrent,...
AdamO's user avatar
  • 63.1k
2 votes
1 answer
837 views

Statistical test to compare models results for concordance index

I'm working on a model for survival prediction and using the concordance index to evaluate the results (https://medium.com/analytics-vidhya/concordance-index-72298c11eac7) . I want to show that my ...
coffecake8284's user avatar
1 vote
0 answers
46 views

Calculating the integrated prediction error curve

I am reading Tutz & Schmid "Modeling Discrete Time-to-Event Data" (2016) chapter 4 Evaluation and Model Choice section 4.3.2 Prediction Error Curves. Integrated prediction error curve is ...
Richard Hardy's user avatar
1 vote
0 answers
160 views

Concordance Correlation Coefficient for negative correlations

A question regarding the Concordance correlation coefficient (CCC). I would like to implement it in Python. Following the Wikipedia entry, one of the possible implementations would be ...
John Titor's user avatar
0 votes
1 answer
70 views

Determine cause of poor concordance index?

I have a survival dataset which I’ve experimented with to create several Cox PH models using different techniques (lasso, forward selection, backwards elimination etc), however no matter which ...
Sam Kennedy's user avatar
3 votes
1 answer
170 views

Extension of the relationship between ROCAUC/c-index and Wilcoxon-Mann-Whitney U

There is a relationship between ROCAUC and the Wilcoxon Mann-Whitney U test of the probability predictions of the two groups. If our $y$ variable has more than two classes, can we extend this to a ...
Dave's user avatar
  • 63.6k
1 vote
1 answer
780 views

How to interpret time-varying AUC for Survival Analysis?

I have trained some models for survival forecast (only interested in ranking of observations). I obtained C-index values as well as dynamic AUC values for the three trained models (CPH, AFT -weibull- ...
BorisD's user avatar
  • 35
2 votes
1 answer
505 views

Why does random survival forest use Cumulative Hazard Function to calculate C Index?

I am working with Random Survival Forests and I found out in the documentation that it uses Sum of Cumulative Hazard Function to define a worse predicted outcome, which is subsequently used to compute ...
napoleon's user avatar
  • 123
0 votes
0 answers
182 views

Comparison of experimental method against gold standard?

I have a method X, assumed to be the gold standard. Then, I have a method Y. The response variable (for example, weight) is a continuous variable. For a sample of n individuals, the response variable ...
BlueSea's user avatar
  • 23
0 votes
1 answer
1k views

Customized concordance index for survival analysis in python

I am trying to build a two-stage model where I classify my data and split it into two datasets. I then fit a Cox Proportional Hazard model to each dataset and make predictions and can calculate the C-...
andKaae's user avatar
  • 163
1 vote
1 answer
839 views

Evaluating fitness of CoxPH survival regression model with Concordance-index

I'm doing a survival analysis for cancer patients. I have two different models for two different types of cancer (different datasets). Some key differences between model 1 and 2 are: Model 1 has only ...
beerzy's user avatar
  • 33
1 vote
1 answer
168 views

Concordance estimate survival data in long format

I would like to use concordance to have an idea of my models discrimination power. Because I have time dependent covariates, the time to event data is in a "long" format, i.e. one row per ...
DURAL's user avatar
  • 21
1 vote
0 answers
238 views

Concordance correlation always less than or equal to pearson correlation

Can someone explain intuitively (or even mathematically) why Lin's concordance correlation coefficient (CCC) is always less than or equal to pearson's correlation coefficient (PCC). A related query ...
Maaz's user avatar
  • 317
1 vote
0 answers
59 views

Is it necessary to do multiple hypothesis testing for AUC comparison? [closed]

Dear all: Is it necessary to do multiple hypothesis testing for AUC comparison. For example, if I calculate AUC from logistic regression, there is an AUC value. As we know, we can compare two models ...
codecrazer's user avatar
2 votes
0 answers
131 views

Can you have a concordance statistic (discrimination) when predicting a continuous outcome?

I'm writing a proposal for a prediction model predicting BP (continuous outcome, predicting trend over time). For assessing model performance, I'm seeing discrimination and calibration as the most ...
rodeoclowns's user avatar
1 vote
1 answer
79 views

circular ranked lists

I have two lists which are circular, in that given list ['A','B','C','D','E'] 'A' and 'E' are the same distance apart as 'A' and 'B' and so on. Given I have rankings R1=['A','B','C','D','E'] R2=['...
ha554an's user avatar
  • 83
2 votes
1 answer
114 views

Cohen's kappa and confounders

I'm interested in how certain factors affect the inter-rater reliability of two driving examiners (sitting in the back seats) in deciding whether to pass or fail the examinee. For example, age, ...
Orion's user avatar
  • 214
0 votes
0 answers
183 views

Does a high negative concordance imply reproducible features

Does a high negative(-1) Lin's concordance correlation coefficient(Lins CCC) imply reproducible features ? I am calculating Lin's CCC of radiomic features between two different CT scan settings , ...
Michael Moran's user avatar
6 votes
0 answers
221 views

What are the pros and cons of different metrics for evaluating a logistic regression model?

In the data science world, I have always evaluated the performance of logistic regression models simply using ROC/AUC. However recently, I've read from some traditional statistics source about some ...
CPT's user avatar
  • 461
4 votes
1 answer
694 views

Theoretical computation Kendall's tau [closed]

While working with rank correlation statistics of samples, specifically Spearman's rho and Kendall's tau, I came across the interesting theoretical interpretation of Kendall's tau. It can be viewed as ...
rieemann's user avatar
3 votes
1 answer
3k views

Use median survival time to calculate CPH c-statistic?

I'm interested in calculating the concordance ($c$-statistic) of a proportional hazards model. For reference, the c-statistic is detailed in Stat Med. 1996 Feb 28;15(4):361-87. The relevant quote ...
John Kleve's user avatar
0 votes
1 answer
1k views

Correct interpretation of Kendall's W

This is a follow-up to my earlier question. Let me summarise what the original question dealt with. I asked whether I could compute Kendall's W for a case where the same question was asked for ...
menteith's user avatar
  • 157
0 votes
1 answer
294 views

Kendall's W for a series of answers about the same phenomenon in different entities

I have a question about the proper usage of Kendall's coefficient of concordance (Kendall's W). Suppose a series of questions were asked how democratic certain organisations are. Each question was the ...
menteith's user avatar
  • 157
7 votes
1 answer
11k views

How to interpret concordance index in Cox models?

I'm new using the survival package in R. Consider an example provided with the package: ...
jroberayalas's user avatar
3 votes
0 answers
342 views

How to manually calculate concordance for a coxPH model

I am struggling to work out how the concordance statistic of a coxPH model is derived. I have built a coxPH model on a small dataset (4 observations). The resultant model is given below: ...
Alex's user avatar
  • 61
3 votes
3 answers
2k views

What is the right measure to compare survival models?

Suppose that there are N patients indexed by n = 1, 2, ..., N. Suppose that I estimate two different survival models M1 and M2. Model $i$ gives a risk score $r_n^i$ for patient $n$. According to ...
I Like to Code's user avatar
2 votes
2 answers
2k views

AUC/C-index confidence interval estimate

Is there a way to compute an estimate of the AUC/C-index confidence interval, simply by using a mean AUC/C-index and sample size?
Héctor van den Boorn's user avatar
5 votes
2 answers
6k views

How does AUC of ROC equal concordance probability? [duplicate]

If this answer on Quora is correct then I think I understand what concordance probability is. However, I also find that this answer on StackExchange provides the formula of concordance probability ...
ntvy95's user avatar
  • 317
2 votes
0 answers
521 views

How to calculate the standard deviation of the C-index?

I fitted a coxph model and below is the concordance details from my object named cx. I can see that ...
Scarper's user avatar
  • 71
3 votes
0 answers
2k views

Concordance index in survival and rms packages

I would like to compute concordance index to validate Cox model. From the following post concordance rms I know that I can use rms package and ...
user1700890's user avatar
2 votes
1 answer
854 views

How to calculate concordant percentage in a validation data

I've created a logistic regression model in R using the glm function using a bank data and. I am able to get the AIC value, coefficients, percentage concordant, tied and discordant pairs in test and ...
Anirban Ghosh's user avatar
1 vote
1 answer
116 views

using non-cancerous patients to predict survival time from cancer?

I have data that consists of lab test results for patients, cancer dates for those who got cancer, and time till death/censor from that cancer. This cancer is both rare (most of the patients never get ...
ihadanny's user avatar
  • 3,340
2 votes
0 answers
152 views

Kendall W for continuous data

Simple question: can I use the Kendall W test for continuous data? Cuz I used it for a bunch of continuous variables and almost all coefficients are very close to 1, with p-values very very very small ...
user1384636's user avatar
2 votes
1 answer
93 views

Is it normal for logistic regression, to have predictors which have good Wald's Chi Sq, but still bad performance?

I have trying to build a logistic model with some 10 variables. All of those variables have Wald Chi sq value<500. All are highly significant p-value<0.0001. Event rate is about 3%. The ...
user2542275's user avatar
0 votes
0 answers
321 views

classification for concordance index

In this paper (and in most of the others I found), the authors want to find a continuous predictor that maximizes the concordance index. I would like to have a binary classifier instead that ...
user1225905's user avatar
1 vote
0 answers
50 views

Can Cohen's kappa be used for raters with different but related targets?

There are two binary raters: Rater 1 is certified to assess the ability of people to drive a taxi. Rater 2 is certified to assess the ability of people to drive a lorry. Rater 1 and 2 have both ...
Orion's user avatar
  • 214
3 votes
1 answer
4k views

How to compare predictive power of survival models?

Frank Harrell describes the concordance (or Somer's D) as not being sensitive enough to compare multiple survival models for their diagnostic ability, and I've observed this in my own work with ...
Stuart Lacy's user avatar
2 votes
0 answers
580 views

Interpreting a concordance < 0.5 in survival analysis

I've seen it detailed that a concordance of < 0.5 can be viewed as a measure of better than random predictive power, similarily to the AUC. When a classifier obtains an AUC of 0.2, you know that it'...
Stuart Lacy's user avatar
4 votes
1 answer
3k views

Inter-(-intra) class correlation coefficient or intra-(inter-) concordance coefficient in python

My question is about how to calculate inter-(intra-) class correlation coefficient (ICC) or intra-(inter-) concordance coefficient (CCC), ideally in python. My dataset consists of several dozen ...
artmnv's user avatar
  • 41
1 vote
2 answers
189 views

Cox regression covariate specific concordance independent of distribution of covariates in sample?

Concordance seems to be a very good metric for telling you how well the model allows you to order individuals for the particular sample that is being used. However, when applying a cox regression ...
Clark's user avatar
  • 215
9 votes
3 answers
6k views

Bland-Altman (Tukey Mean-Difference) plot for differing scales

I find that Bland-Altman plots for comparing two methods are extremely useful in assessing agreement. However, I'm curious if there is a similar method or transformation that can be used when the ...
Ashe's user avatar
  • 1,135
3 votes
2 answers
366 views

Quantifying consistency between two groups of raters

I have two groups of people, group A (62 people) and group B (8 people), and each person in each group responded to a test with the same 8 questions where the answer to each question was one of the ...
derNincompoop's user avatar
3 votes
1 answer
1k views

Does concordance correlation require data to be normally distributed?

I would like to know if the results from Device 'A' and Device 'B' are in agreement when measuring the same thing. I have 258 pairs of results. The results are interval variables and have the same ...
yun's user avatar
  • 31