Questions tagged [method-comparison]
Questions about comparisons of different statistical methods
145
questions
0
votes
0
answers
10
views
Comparing Sensitivity, Specificity, PPV and NPV of one test across two subgroups
I am seeking to compare the diagnostic performance of a test within a population with two defined subgroups.
I have the raw data & computed data, but am unsure what the best test is to compare the ...
- 1
0
votes
0
answers
23
views
Uncertainty in slope
I've created a linear relationship between two different measurements. The equation is in the form of y=mx and I am trying to validate my slope within a certain range so that when I run my experiment, ...
- 1
0
votes
0
answers
12
views
Kappa coefficient, prevalence and bias. How to interpret wide confidence intervals, some falling outside [-1,1]
I an assessing the agreement between two measurements methods on a binary presence/absence metric.
Here is one example of a contingency table:
...
- 197
0
votes
0
answers
6
views
Statistical Test for comparing two exponential decay fits
in my research, I have two datasets containing two parameters (time, concentrations) for two conditions (low pH, high pH). I have fitted the data points of each condition with an exponential decay ...
0
votes
0
answers
23
views
Method comparison: removing double-zeros before orthogonal regression, deming or passing-bablock?
I am comparing two pieces of equipment / methods which are supposed to measure the same information over time.
I looked into the literature and decided Passing-Bablock seemed like the right approach (...
- 197
0
votes
0
answers
14
views
Comparing proportions in dynamic populations
I have an example of dataset like this:
Members of insurance plan
01.01.2021
01.01.2022
<18 y. o
2100
3000
Total members
200000
230000
How can I compare proportions and define if proportion on ...
- 11
2
votes
0
answers
18
views
Best statistical model to compare student metrics
I want to understand what would be the best statistical model to use if I'm comparing students basis their metrics that carry a certain weightage. Details are as below
Metric (Weightage)
% of ...
- 21
0
votes
0
answers
40
views
Analyzing method comparison studies that include repeated measurements
I am currently a graduate student working on my thesis, trying to analyze method comparison studies that include repeated measurements. I have a dataset created in the context of a method comparison ...
- 1
0
votes
0
answers
24
views
A/B testing and Predictive modelling - Subscription models
My problem is kind of a business case. I doing an analysis for an expiration plan. The current plan is 3 months, and the statistics are not so good. The conversion rate is on the decline, say 2%/month....
0
votes
0
answers
29
views
Different ways to do autoregression in R
Can someone explain why different ways of doing autoregression give different estimates? For example, I can use regression with a lag, or I can use arima(), or gls() in package nlme:
...
0
votes
1
answer
66
views
Which distribution to use ? Justify the choice and find a criterion of compatibilty between expriment and theoritical values [closed]
In an experiment, 16 events corresponding to a given physical process were measured. The theoretical model predicts the observation of $3.9$ events.
I am looking for the probability that the ...
2
votes
0
answers
27
views
Correct hypothesis test to use here?
I have 5 samples from which I gathered data at 5 separate time points (days).
Each day I observe the distribution of some RV.
When I plot the distribution for each day it sort of looks like this for ...
- 21
0
votes
1
answer
31
views
Age adjustment or Age standardization state of the art?
I want to run some comparisons of a quantity between different populations. Such quantity comes from a model that includes, among other variables, the variable age. So, one of the variables affecting ...
- 1
1
vote
2
answers
476
views
Comparing frequency between groups
I'm using SPSS, new to statistics.
I have 2 groups of patients (Group 1: 1000 patients; Group 2: 400 patients).
One of five different interventions (surgery a,b,c,d,e) was performed on subjects in ...
- 11
0
votes
0
answers
83
views
How to compare/test two datasets which are not supposed to be exactly equal
Example of such dataset is open, high, low, close, volume data of any stock over the period of last 5 days. This data is available at the frequency of each minute. I want to test the quality of my ...
- 101
2
votes
2
answers
68
views
Comparing results from structural equation model and OLS regressions
Are results obtained from a structural equation model (SEM) comparable to those from a series of separate regressions?
Or, conversely, can I obtain direct, indirect, and total effect of an independent ...
- 1,007
0
votes
1
answer
128
views
anova model comparison versus concordance index comparison in deciding best model for survival prediction
I have developed a cox-model in my research, and I want to compare it to the conventional staging system for prognostic evaluation. The traditional staging system categorizes patients into risk groups ...
- 91
1
vote
0
answers
76
views
GLM Model comparison and selection with different response distribution
I am proposing for my thesis... Pls help.
I have Y=weight of Baby, X1=Age of Mother X2=Age of Baby and X3=Sex of baby.
I assume Y~Normal with Identity link.
Also
I assume Y~Binomial with logit link
...
- 11
2
votes
1
answer
410
views
The reason (and intuition) behind why stochastic gradient descent can get stuck on a local minimum
Suppose you want to find $k$ that minimises your cost function $J(k)$. We may want to apply batch gradient descent or stochastic gradient descent. Let's deliberately initialise $k$ with the same ...
- 521
0
votes
0
answers
310
views
How to compare 2 p-values?
I have 2 continuous outcome (independent) variables, A and B, and 1 dependent variable (biomarker) that are all very correlated (i.e., p_A = 0.01, p_B= 0.000001). I would like compare the outcome ...
- 73
0
votes
1
answer
167
views
How to compare two texts with different order of words?
I have two texts, one ground truth and one OCR result, and I want to measure to what accuracy the result matches the ground truth. But since the text source is non-linear, both texts have a different ...
- 1
11
votes
2
answers
2k
views
Is it wrong to compare multiple models on the same test set and choose the best model?
Suppose we split a dataset into 3 parts (train, validation, and test). I know that it's important to make sure the test set doesn't influence our decisions during model selection or hyperparameter ...
1
vote
1
answer
181
views
Contour plot comparison [closed]
What are the ways to compare two different maps? I was thinking of taking the matrices of each map and then quantify their spatial differences by plotting the map of the differences of the two ...
- 13
2
votes
2
answers
227
views
Fair comparison of machine learning methods with different parameterization
Let us assume the following scenario: The goal is to forecast the additional number of customers of a set of companies. To do this, you use a basic set of characteristics (e.g., company-specific ...
- 73
1
vote
2
answers
1k
views
McNemar-Bowker exact test in R: difficulty interpreting the results
I am trying to use the McNemar-Bowker test to test the difference in performance for 2 classifiers. Since my input matrix is sparse and the sum of some of the symmetric cells is less than 10, I am ...
- 23
3
votes
1
answer
98
views
Comparing the accuracy of binary classifiers using iterated cross-validation
Let's say that you want to compare two binary classifiers (e.g., LDA and linear SVM) for a given research question, the question being "which one will probably perform best for the problem at ...
- 289
1
vote
1
answer
34
views
Overlapping quinquennial data - How to compare?
I actually do not know if this is the right place to ask, but I hope someone can help me.
I have a dataset that provides the number of newborns in quinquennial periods. For instance, I have the number ...
- 13
1
vote
1
answer
522
views
Do XBGoost and LightGBM only use trees as base learners?
From my understanding of GBM is that it can take not only decision trees as base learners, but different weak learners as well (e.g., linear models), since it relies on sequential gradient descent. ...
- 27
1
vote
2
answers
417
views
Comparison of Bayesian and Classical estimates
Is it correct to compare Bayesian and Classical estimates using Mean Squared Error (MSE)? MSE is a criterion that is used in the classical paradigm.
For example: I am comparing the performance of ...
- 11
1
vote
0
answers
49
views
Which regression analysis to use?
I have percentage change data compared to a baseline of public and private transport activity over 1 year. With the help of one or two independent variables I want to analyze if there is a switch ...
- 11
2
votes
5
answers
224
views
Interval Estimation for a Change in a Binomial Proportion
I'm not sure how to estimate the confidence interval (CI) for a change in a small sample size binomial proportion using the same sample set both times.
I have two methods that I would like to compare (...
- 179
0
votes
1
answer
35
views
Statistical Significance on shops performances with a marketing promotion
I have data regarding the performances of shops on a certain time period: in this data some shops had an active marketing promotion, and some didn't. I would like to understand whether the shops with ...
- 1
0
votes
1
answer
601
views
Comparing two clustering methods with adjusted Rand index - How to make labels match
I have a set of data (countries), and I use a clustering method (Kmeans) to create two partitions on them based on different data (diet and covid stats). So I get two groups of partitions, and I want ...
- 1
11
votes
2
answers
491
views
Propensity score matching vs non-parametric regression
I am trying to understand the benefit of propensity matching over non-parametric regression for causal inference from non-experimental data.
As background: the way I understand it, parametric ...
- 113
1
vote
0
answers
39
views
Efficient way to compare algorithms performance with missing samples
I'm currently doing some experiments to compare more than 70 different configurations (different parameters of an evolutionary algorithms) that have 30 runs each (repeated due to stochastic results). ...
3
votes
1
answer
886
views
What is the difference between simulated annealing and deterministic annealing?
Not sure if this is the right place, but I was wondering if someone could briefly explain to me the differences & similarities between simulated annealing and deterministic annealing?
I know that ...
- 788
1
vote
2
answers
1k
views
How can I tell if images are from the same distribution?
What are some tests I could run on two groups of imagery to see if they're from the same distribution? Imagine, for example, that you want to augment a dataset with another, and you want to make sure ...
- 398
1
vote
0
answers
2k
views
R: Caret package: AUC ("ROC") standard error from cross validation
This seems like a basic questions, so I'm likely missing the big picture here...
I would like to know the AUC of each fold of a cross-validation performed in Caret's train function, in order to ...
- 111
0
votes
1
answer
74
views
Statistical tests for comparing (black box) model performance on two datasets
Introduction
I have two datasets, dataA (real images) and dataB (generated images). I also have a model that has been trained on dataC. dataC might have some overlap with dataA but for certain does ...
- 3
3
votes
1
answer
799
views
Comparing different model with the same output
I work on developing 3 different models for predicting land surface temperatures in an area (validation is done with images taken by a thermal drone). The models are:
A thermodynamic mechanistic ...
7
votes
2
answers
160
views
What is better in Monte Carlo integration: product of means or mean of products?
Let $X$ and $Y$ be two independent continuous random variables with pdfs $f_X$ and $f_Y$, respectively. Let $\varphi_1$ and $\varphi_2$ be two continuous functions from ${\mathbb R}$ to ${\mathbb R}$. ...
- 73
0
votes
0
answers
67
views
why the result of primal sim is not the same as linear kernel?
I have a data set with 36000 rows and 9 columns. so n<< m. this is multi classification SVM. I solved this by primal model in OSQP. then I used R package e710 and svm with linear kernel.
the ...
- 137
2
votes
1
answer
349
views
When to use the full and the conditional likelihood
In the context of estimating parameters of a time series model, we may consider either the full likelihood or the conditional likelihood. I was wondering
in what situations the full ...
- 323
0
votes
2
answers
43
views
Evaluate methods across datasets based on distance metrics
I want to evaluate $n$ methods, says $M_1,\cdots,M_n$ on $m$ datasets $D_1, \ldots, D_m$. For each method $M$, I have a score function $f(M,D) \in [0, \infty)$ for dataset $D$. A perfect method will ...
- 111
1
vote
1
answer
21
views
if different methods have difference 0.5 in third decimal place while finding accuracy then what to conclude?
For the comparison of different feature selection methods and evaluating performance metrics. After evaluation when their is only 0.5 difference in third decimal place. Is this difference is ...
2
votes
1
answer
1k
views
Which method is correct for calculating total deviance explained for boosted regression trees?
I have the following output from a boosted regression trees model and I would like to calculate the total deviance explained.
...
- 93
1
vote
1
answer
3k
views
Can you compare the mean of one data set with the median of another?
I’m having an argument with another numerically literate person. We both saw a news site state that under no circumstances should you compare the median of one set with the mean of another.
It’s ...
2
votes
0
answers
67
views
comparison of machine learning models
I’m struggling with a statistical problem in machine learning:
I’m using event-related fMRI responses (voxels) evoked by graded painful stimulations as feature. The targets are the scores (labels) of ...
- 71
3
votes
2
answers
1k
views
Reasons for different parameters via MoM and MLE
I got a study of 210 samples and I tried fitting gamma distribution to them. I used method of moments and maximum likelihood estimation to calculate the parameters, but parameters came out quite ...
- 163
1
vote
0
answers
8
views
Evaluating a set of algorithms based on daily performance
I work in a production system which can use three competing algorithms, say $A_1,A_2,A_3$ for a customer on a daily basis. On each day, for a given customer, the production system looks at how much ...
- 251