Questions tagged [method-comparison]

Questions about comparisons of different statistical methods

Filter by
Sorted by
Tagged with
0 votes
0 answers
16 views

for method comparison studies do we ever use randomisation, period effects and washout periods?

By method comparison I mean we have a number of "individuals" and for each a quantity eg blood pressure is measured by 2 measuring devices. So we have a pair of measures for each person 1 ...
user3156942's user avatar
2 votes
1 answer
34 views

method comparison/agreement - is Bland-Altman or equivalence of the mean best

I am interested in the appropriate technique for assessing the agreement of the paired values of measurements made by 2 measuring devices, an equivalence test of the mean of the difference in the ...
user3156942's user avatar
0 votes
0 answers
20 views

Estimating Standard Error of the Mean Across Multiple Samples: Method Selection and Best Practices

I have multiple samples, each of size (n), and I'm exploring different methods to estimate the Standard Error (SE) of the mean. I understand there are several approaches, but I'm seeking insights into ...
Catfoxes's user avatar
0 votes
0 answers
19 views

Statistical analysis (comparison) of high-frequency timeseries data for two temperature loggers?

I am working with high-frequency (144 measurements/day) temperature data. It was two individual temp instruments measuring at equal intervals (10 mins) at the same stationary location for 120 days. ...
R Reid's user avatar
  • 1
0 votes
1 answer
40 views

Best statistical method to compare model data with data from real measurements

I am relatively new to statistics therefore my question may be a bit silly, but I would like to compare data (soil water content) obtained from a mathematical model with data obtained from real ...
Skamby's user avatar
  • 1
2 votes
0 answers
33 views

Is there a statistic that expresses whether two measures tell us the same information?

After trying to understand different implementations of intraclass correlation (ICC), inter-rater reliability, and bland-altman statistics, I'm not quite sure what is the right choice here, and would ...
stck8888's user avatar
4 votes
0 answers
49 views

What analysis should I use to compare two groups on count data, non-independent observations?

I need some help to figure out how to choose the appropriate statistical analysis, it's driving me crazy. We are conducting a service evaluation, where we have two different groups who voted on their ...
Luca's user avatar
  • 41
0 votes
0 answers
20 views

Comparing Age-Adjusted Mortality Rate between 2 groups

Context: I am using national mortality database to compare the mortality of X disease by demographic factors. I divided the county based on social vulnerability scoring into 4 groups (1,2,3,4 based on ...
OlgaSan's user avatar
2 votes
1 answer
96 views

Best method to compare measurement techniques

I am wishing to test the equivalence, or not of two measurement methods. Some background, we have two test instruments (old and new) that measure radiation dose distributions compared to what was ...
James's user avatar
  • 23
2 votes
1 answer
70 views

Comparison of Data that looks at correlation and absolute values

I am looking for a way to compare two sets of data in order to find out how similar they are to each other. My application: I try to compare multiple Measurement methods that both measure the sound ...
Anton Wolf's user avatar
0 votes
0 answers
33 views

Method comparison studies - which method is "better"?

I have two laboratory methods to measure the same property. One method is principally instrument based the other is more of a manual method. I believe in my heart that the instrument-based method is ...
Jared Johansen's user avatar
1 vote
0 answers
20 views

Comparing two populations with different amount of 'categories'. Need to know by how much (%) 'category' A has increased [closed]

Very new to statistics and I don’t know the correct terms or what to Google! In 2021 x amount of visas were refused for 5 different reasons. In 2022 y amount of visas were refused for 6 different ...
A.Herron's user avatar
0 votes
0 answers
9 views

Validating a new definition for detection

I am having some trouble understanding what approach I would use to validate a new set of definitions versus the "gold standard" to detect disease positivity. For example, if I wanted to say ...
Levi M's user avatar
  • 75
0 votes
1 answer
39 views

Chi-square: correlation or comparison test?

I find some references and discussion state that Chi-square test used for comparison analysis, but some said for correlation also. I am confused, so Chi-square is test for correlation or comparison? ...
dheni koerniawan's user avatar
1 vote
0 answers
33 views

Comparing Sensitivities of Microscopic and Macroscopic Models (Travel Demand)

A simulation output for each microscopic travel demand model and two further macroscopic MDCEV models - transformed versions of the microscopic model - are given for a base scenario. Moreover, three ...
meg1234's user avatar
  • 31
0 votes
0 answers
130 views

Comparing Sensitivity, Specificity, PPV and NPV of one test across two subgroups

I am seeking to compare the diagnostic performance of a test within a population with two defined subgroups. I have the raw data & computed data, but am unsure what the best test is to compare the ...
Richard's user avatar
1 vote
0 answers
45 views

Kappa coefficient, prevalence and bias. How to interpret wide confidence intervals, some falling outside [-1,1]

I an assessing the agreement between two measurements methods on a binary presence/absence metric. Here is one example of a contingency table: ...
Timelate's user avatar
  • 309
0 votes
0 answers
17 views

Comparing proportions in dynamic populations

I have an example of dataset like this: Members of insurance plan 01.01.2021 01.01.2022 <18 y. o 2100 3000 Total members 200000 230000 How can I compare proportions and define if proportion on ...
Vlad Fedo's user avatar
2 votes
0 answers
22 views

Best statistical model to compare student metrics

I want to understand what would be the best statistical model to use if I'm comparing students basis their metrics that carry a certain weightage. Details are as below Metric (Weightage) % of ...
Jonas's user avatar
  • 21
0 votes
1 answer
76 views

Analyzing method comparison studies that include repeated measurements

I am currently a graduate student working on my thesis, trying to analyze method comparison studies that include repeated measurements. I have a dataset created in the context of a method comparison ...
kjcb's user avatar
  • 1
0 votes
0 answers
39 views

A/B testing and Predictive modelling - Subscription models

My problem is kind of a business case. I doing an analysis for an expiration plan. The current plan is 3 months, and the statistics are not so good. The conversion rate is on the decline, say 2%/month....
Lukest_Dang's user avatar
0 votes
0 answers
33 views

Different ways to do autoregression in R

Can someone explain why different ways of doing autoregression give different estimates? For example, I can use regression with a lag, or I can use arima(), or gls() in package nlme: ...
Richard G's user avatar
2 votes
0 answers
29 views

Correct hypothesis test to use here?

I have 5 samples from which I gathered data at 5 separate time points (days). Each day I observe the distribution of some RV. When I plot the distribution for each day it sort of looks like this for ...
YYH's user avatar
  • 21
0 votes
1 answer
36 views

Age adjustment or Age standardization state of the art?

I want to run some comparisons of a quantity between different populations. Such quantity comes from a model that includes, among other variables, the variable age. So, one of the variables affecting ...
Compare's user avatar
1 vote
2 answers
1k views

Comparing frequency between groups

I'm using SPSS, new to statistics. I have 2 groups of patients (Group 1: 1000 patients; Group 2: 400 patients). One of five different interventions (surgery a,b,c,d,e) was performed on subjects in ...
user350382's user avatar
0 votes
0 answers
121 views

How to compare/test two datasets which are not supposed to be exactly equal

Example of such dataset is open, high, low, close, volume data of any stock over the period of last 5 days. This data is available at the frequency of each minute. I want to test the quality of my ...
KnowledgeSeeeker's user avatar
2 votes
2 answers
129 views

Comparing results from structural equation model and OLS regressions

Are results obtained from a structural equation model (SEM) comparable to those from a series of separate regressions? Or, conversely, can I obtain direct, indirect, and total effect of an independent ...
Fuca26's user avatar
  • 1,097
1 vote
1 answer
248 views

anova model comparison versus concordance index comparison in deciding best model for survival prediction

I have developed a cox-model in my research, and I want to compare it to the conventional staging system for prognostic evaluation. The traditional staging system categorizes patients into risk groups ...
Bent Larsen's user avatar
2 votes
0 answers
101 views

GLM Model comparison and selection with different response distribution

I am proposing for my thesis... Pls help. I have Y=weight of Baby, X1=Age of Mother X2=Age of Baby and X3=Sex of baby. I assume Y~Normal with Identity link. Also I assume Y~Binomial with logit link ...
A M Bello's user avatar
3 votes
1 answer
677 views

The reason (and intuition) behind why stochastic gradient descent can get stuck on a local minimum

Suppose you want to find $k$ that minimises your cost function $J(k)$. We may want to apply batch gradient descent or stochastic gradient descent. Let's deliberately initialise $k$ with the same ...
mathgeek's user avatar
  • 541
0 votes
0 answers
613 views

How to compare 2 p-values?

I have 2 continuous outcome (independent) variables, A and B, and 1 dependent variable (biomarker) that are all very correlated (i.e., p_A = 0.01, p_B= 0.000001). I would like compare the outcome ...
Sirius Hou's user avatar
0 votes
1 answer
253 views

How to compare two texts with different order of words?

I have two texts, one ground truth and one OCR result, and I want to measure to what accuracy the result matches the ground truth. But since the text source is non-linear, both texts have a different ...
Shanksum's user avatar
14 votes
2 answers
4k views

Is it wrong to compare multiple models on the same test set and choose the best model?

Suppose we split a dataset into 3 parts (train, validation, and test). I know that it's important to make sure the test set doesn't influence our decisions during model selection or hyperparameter ...
Abdulwahab Almestekawy's user avatar
1 vote
1 answer
394 views

Contour plot comparison [closed]

What are the ways to compare two different maps? I was thinking of taking the matrices of each map and then quantify their spatial differences by plotting the map of the differences of the two ...
Random's user avatar
  • 13
2 votes
2 answers
301 views

Fair comparison of machine learning methods with different parameterization

Let us assume the following scenario: The goal is to forecast the additional number of customers of a set of companies. To do this, you use a basic set of characteristics (e.g., company-specific ...
salomon's user avatar
  • 73
1 vote
2 answers
2k views

McNemar-Bowker exact test in R: difficulty interpreting the results

I am trying to use the McNemar-Bowker test to test the difference in performance for 2 classifiers. Since my input matrix is sparse and the sum of some of the symmetric cells is less than 10, I am ...
ILR's user avatar
  • 23
4 votes
1 answer
123 views

Comparing the accuracy of binary classifiers using iterated cross-validation

Let's say that you want to compare two binary classifiers (e.g., LDA and linear SVM) for a given research question, the question being "which one will probably perform best for the problem at ...
Philopolis's user avatar
1 vote
1 answer
35 views

Overlapping quinquennial data - How to compare?

I actually do not know if this is the right place to ask, but I hope someone can help me. I have a dataset that provides the number of newborns in quinquennial periods. For instance, I have the number ...
Gigi39's user avatar
  • 13
1 vote
1 answer
839 views

Do XBGoost and LightGBM only use trees as base learners?

From my understanding of GBM is that it can take not only decision trees as base learners, but different weak learners as well (e.g., linear models), since it relies on sequential gradient descent. ...
user000's user avatar
  • 37
1 vote
2 answers
731 views

Comparison of Bayesian and Classical estimates

Is it correct to compare Bayesian and Classical estimates using Mean Squared Error (MSE)? MSE is a criterion that is used in the classical paradigm. For example: I am comparing the performance of ...
rgt's user avatar
  • 11
2 votes
5 answers
236 views

Interval Estimation for a Change in a Binomial Proportion

I'm not sure how to estimate the confidence interval (CI) for a change in a small sample size binomial proportion using the same sample set both times. I have two methods that I would like to compare (...
R. Cox's user avatar
  • 179
0 votes
1 answer
46 views

Statistical Significance on shops performances with a marketing promotion

I have data regarding the performances of shops on a certain time period: in this data some shops had an active marketing promotion, and some didn't. I would like to understand whether the shops with ...
Luckyna's user avatar
1 vote
2 answers
993 views

Comparing two clustering methods with adjusted Rand index - How to make labels match

I have a set of data (countries), and I use a clustering method (Kmeans) to create two partitions on them based on different data (diet and covid stats). So I get two groups of partitions, and I want ...
Pythalex's user avatar
11 votes
2 answers
729 views

Propensity score matching vs non-parametric regression

I am trying to understand the benefit of propensity matching over non-parametric regression for causal inference from non-experimental data. As background: the way I understand it, parametric ...
Shade's user avatar
  • 113
1 vote
0 answers
41 views

Efficient way to compare algorithms performance with missing samples

I'm currently doing some experiments to compare more than 70 different configurations (different parameters of an evolutionary algorithms) that have 30 runs each (repeated due to stochastic results). ...
Laurent Parmentier's user avatar
3 votes
1 answer
1k views

What is the difference between simulated annealing and deterministic annealing?

Not sure if this is the right place, but I was wondering if someone could briefly explain to me the differences & similarities between simulated annealing and deterministic annealing? I know that ...
Tinu's user avatar
  • 828
1 vote
2 answers
2k views

How can I tell if images are from the same distribution?

What are some tests I could run on two groups of imagery to see if they're from the same distribution? Imagine, for example, that you want to augment a dataset with another, and you want to make sure ...
jss367's user avatar
  • 408
1 vote
0 answers
3k views

R: Caret package: AUC ("ROC") standard error from cross validation

This seems like a basic questions, so I'm likely missing the big picture here... I would like to know the AUC of each fold of a cross-validation performed in Caret's train function, in order to ...
StinkySocks's user avatar
0 votes
1 answer
102 views

Statistical tests for comparing (black box) model performance on two datasets

Introduction I have two datasets, dataA (real images) and dataB (generated images). I also have a model that has been trained on dataC. dataC might have some overlap with dataA but for certain does ...
krtlfs's user avatar
  • 3
3 votes
1 answer
1k views

Comparing different model with the same output

I work on developing 3 different models for predicting land surface temperatures in an area (validation is done with images taken by a thermal drone). The models are: A thermodynamic mechanistic ...
Ronny Efronny's user avatar