Stack Exchange Network

Stack Exchange network consists of 175 Q&A communities including Stack Overflow, the largest, most trusted online community for developers to learn, share their knowledge, and build their careers.

Visit Stack Exchange

Questions tagged [method-comparison]

Questions about comparisons of different statistical methods

1
vote
1answer
12 views

if different methods have difference 0.5 in third decimal place while finding accuracy then what to conclude?

For the comparison of different feature selection methods and evaluating performance metrics. After evaluation when their is only 0.5 difference in third decimal place. Is this difference is ...
0
votes
0answers
22 views

How to perform image analysis on different dataset? [closed]

I have two different datasets containing raw cells images of microscopic slides, the two datasets contains images from a bunch of slides processed with two different tecniques. The current procedure ...
0
votes
0answers
33 views

Which method is correct for calculating total deviance explained for boosted regression trees?

I have the following output from a boosted regression trees model and I would like to calculate the total deviance explained. ...
1
vote
1answer
33 views

Can you compare the mean of one data set with the median of another?

I’m having an argument with another numerically literate person. We both saw a news site state that under no circumstances should you compare the median of one set with the mean of another. It’s ...
0
votes
0answers
22 views

How to compare two different sized datasets in same interval?

We have made two stress-strain testw (a test which basically measures how much a metal elongates depending on applied force) on two samples taken from same metal. However 2 samples were tested on two ...
2
votes
0answers
59 views

comparison of machine learning models

I’m struggling with a statistical problem in machine learning: I’m using event-related fMRI responses (voxels) evoked by graded painful stimulations as feature. The targets are the scores (labels) of ...
3
votes
2answers
106 views

Reasons for different parameters via MoM and MLE

I got a study of 210 samples and I tried fitting gamma distribution to them. I used method of moments and maximum likelihood estimation to calculate the parameters, but parameters came out quite ...
1
vote
0answers
6 views

Evaluating a set of algorithms based on daily performance

I work in a production system which can use three competing algorithms, say $A_1,A_2,A_3$ for a customer on a daily basis. On each day, for a given customer, the production system looks at how much ...
2
votes
1answer
30 views

Multiple classifier comparison with Bayesian statistics

I've been recently working on some classifier comparisons. I've come to realize, the critical distance diagrams introduced $\approx$ 10 years ago might not be the most relevant solution for this ...
2
votes
0answers
193 views

Compare clustering results with different attributes and number of clusters

I used K-means to cluster a large data set that has millions of samples. I tried to create the clusters with different sets of attributes, which, as a result, generated different optimal number of ...
1
vote
0answers
47 views

Statistical difference of machine learning predictions

I am applying a machine learning classification method to two difference scenarios for my boss (in this context, I am using a random forest). Due to the nature of the problem, there is high class ...
1
vote
0answers
12 views

Comparing Importance of Values to Importance in Change of Values [closed]

I am working with some hourly weather data and I am trying to find how strongly variables correlate and which variables are more important that others. Along with the actual weather data (temperatures ...
2
votes
0answers
41 views

Analysis of Pre-post Likert-type scales: Intervention vs. Control group

I gathered the following data: Intervention group (n= 24, received treatment): pre- and post- scores of different psychometric questionnaires Control group (n=23, inactive): same questionnaires, same ...
1
vote
0answers
26 views

How to compare my model's performance with that of others?

I'm building an audio classifier and would like to compare my model's performance with that of others. I use the Urban Sound Dataset. I searched for already existing results on this dataset but couldn'...
1
vote
0answers
57 views

comparison of two estimators

Assume we have a data set $\mathbf{x}_{n} = (x_{1}, \dots, x_{n})$. Let $\delta_{1}(\mathbf{x}_{n})$ and $\delta_{2}(\mathbf{x}_{n})$ be two consistent estimators of some parameter $\theta \in R^{k}$. ...
2
votes
2answers
300 views

How to compare two levels of one factor

I´ve run a GLMM model with the following variables: Response variable: continuous (with positive, negative and zero values; Explanatory variable: 1 factor with 6 levels (DB, DF, NDB, NDF, BB, BF); ...
6
votes
3answers
248 views

Clarification on how to perform nested cross-validation

I'm currently trying to get lifetables for our population based on death and population counts. My first idea was to follow this paper methodology but after some discussions we are planning to use ...
5
votes
2answers
251 views

Taxonomy/overview of machine learning techniques [duplicate]

My question: I'm looking for a taxonomy/bestiary/overview of machine learning techniques. I would like to learn 1) how these methods relate to each other, and 2) the relative costs and benefits (and ...
1
vote
1answer
1k views

Which method to use to compare two sets of data for similarity? [closed]

I need to validate that my simulation results are correct. So I have two sets of data, my simulation results, and calculations I obtained from an established commercial software to compare against. ...
0
votes
0answers
8 views

Comparing a traditional and technology based tool

I am trying to compare a printed word dictionary with a digital dictionary that included added features like sound and video feedback. For this I have selected 60 participants and asked 5 questions (...
0
votes
1answer
36 views

Model comparing with statistical test

I am comparing few classification models (performance in AUC of ROC curves). I uses 10 times repeated 5 fold cross-validation so I have 50 values of AUC for each model. The key information is, that ...
1
vote
0answers
95 views

Ordered probit model prediction: why highest probabilities and not number of thresholds exceeded?

I'm running ordered probit regression models with polr in R. This question states that for prediction, ...
3
votes
1answer
93 views

Most widely accepted method of SVM model validation?

I am building an SVM based classifier (libsvm) for two distinct balanced classes which are non-linear but separable with an RBF kernel. Both classes have 175 instances each, which must be utilized ...
0
votes
2answers
47 views

How to check significant change in the actual population, not sample [duplicate]

I'm trying to understand how I would decide whether a policy actually had a meaningful impact by comparing the population itself, not samples. An example that comes to mind is comparing the number of ...
0
votes
0answers
147 views

Method(s) to quantify the hysteresis of a data set

I have a data set for two different mixtures of a material with 5 and 6 samples in each data set. I would like to quantify the hysteresis each sample in a "generally accepted" way. The following is my ...
4
votes
1answer
1k views

Most well-known set-similarity measures?

I know of the Jaccard index and the Sørensen-Dice coefficient for computing set similarity, but have been unable to find any other algorithms related to set similarity. This site contains quite a few ...
2
votes
1answer
1k views

Generating Random Data Sets for Linear Regression with Random Slope and Error Term in R

I want to test the effects of sample size on Deming regression using simulated paired data in R. As the data are paired, the expected slope value should be 1 and the intercept 0. The code I have ...
1
vote
0answers
27 views

What statistical analysis to use to compare two measurment tools of creativity?

Participants took two different creativity assessment test. I al trying to make a comparison between the two measurment tools and see which one gives more insight about the level of creativity. What ...
1
vote
0answers
100 views

Specifying the Error Ratio in Deming Regression

We use Deming Regression for method validation in our clinical ELISA laboratory. As we run clinical trials this needs to be performed if we get a new reagent batch, batch of plates, new plate reader ...
0
votes
0answers
11 views

Validation about chosen graph showing rates with big gap between them

I'm trying to show the variation in time of 4 variables, in which the first one is in the order of 20-30 and the remainder are all smaller than 2. The growth rates for each are: 52, 70, 8.85 and 19.06....
1
vote
0answers
103 views

weighted sampling from posterior distribution (i.e. regression predictions)

I am comparing measurement instruments A and B to assess their agreement on measuring feature x. Measurements with A are cheap. I have made measurements on thousands of randomly selected objects with ...
0
votes
1answer
85 views

How to compare effectiveness of methods of parameter estimation on the small samples?

Abstract example. Let's say, I want to compare MLE (Maximum Likelihood Estimation) and LSE (Least Squares Estimation) performance on the small samples for simple linear regression. My idea: to ...
1
vote
1answer
18 views

Comparing difference of means due to individual variability

So in our statistical test, we are using a dependent t-test because we are comparing the means between pre and post conditions in a physiology lab. Since there is a lot of variability between two ...
0
votes
1answer
498 views

Determine correct statistical algorithm for mixed-type data

I have a dataset consisting of ~20000 lines and 8 columns. It is a mixture of binary and float data, but one binary variable is the important one and I want to create a model to predict the other ...
1
vote
0answers
577 views

How to obtain the same results of a random forest model using caret and randomForest?

I am trying to understand how does building a regression model with caret's train () function differs from randomForest(). For my excercise, I am using the iris dataset. As shown in the code below, I ...
1
vote
0answers
231 views

A good metric to simultaneously compare both mean and variance

I am new to statistics, and have a fairly basic question. I have two sets of data that I want to compare with a third dataset. I am only interested in comparing the means and variances. Turns out ...
5
votes
1answer
146 views

Visualizing differences between alternative clusterings?

Does anyone know of a good way to visualize differences between alternative clusterings? I know there are some measures which can be used to quantify the similarity between two partitionings, e.g.: ...
1
vote
3answers
5k views

how can i compare two groups of data?

I asked a question here how can i see if my data are coming from two different population? and seems like I cannot correctly communicate and the person who is writing to me is making me confused. So ...
1
vote
0answers
49 views

What is the difference between jackknifing and LOOCV? [duplicate]

Leave-one-out cross-validation model technique is very similar to jackknifing resampling, because both omitting each training case and perform retraining of the network on the left-out subset. On ...
1
vote
0answers
61 views

What are the benefits of a “deep” over a “shallow” architectures? [duplicate]

What are the benefits of a deep neural network architecture over a shallow architecture with many computational elements. What are pros and cons of these two architectures and its limitations?
0
votes
1answer
102 views

Determining a cut-off value from two populations of unequal sample sizes

I have two groups (healthy and disease, different sample sizes) of data, pooled from the individual subject's measurements of a certain region of the brain. The size of the brain region may be ...
1
vote
0answers
29 views

Statistical comparison between two stochastic algorithms

I am working on a mechanics problem with variability in material properties. For that I need to analyze the efficiency of two methods for same accuracy (confidence level I guess?), the first one being ...
0
votes
1answer
84 views

Need a comparison method to compare original data values and data values derived from a method

I have a gene expression dataset for which I have the original data values and I have put this dataset through a few methods to test their accuracy of prediction. I need to compare the results from ...
3
votes
1answer
98 views

Can an independent t-test be used on paired data when the pairing is unknown?

Suppose the effectiveness of a training course is examined, and performance of each individual in a group is taken both before and after, and the differences are compared in a paired $t$-test. Would ...
3
votes
1answer
63 views

Correspondence between two different depression scales?

I have become involved in a project where psychiatrists are studying the correspondence between two scales measuring the level of depression among a specific patient group. One of the scales is for ...
4
votes
1answer
6k views

Minimum “recommended” sample size for boxplots? Boxplots for different sample sizes

What is the minimum "recommended" sample size to generate boxplots? If I'm comparing different methods and each method has a different sample size, is it fine to use boxplots for this comparison? If ...
3
votes
2answers
2k views

SciPy chisquare applied on continuous data?

chi2 seems to be used to compare distributions between categorical features. Is it possible to use SciPy chisquare function to compare two distributions of continuous data ? For example : ...
2
votes
1answer
102 views

Best method to compare indexes

I have implemented three different estimates to measure the same character, and I would like to statistically compare them (to see if they indeed measure the exact same signal, or if they give ...
1
vote
0answers
452 views

Can you extrapolate values of the dependent variable with a GAM?

I'm trying to find issues where GLMs are better than GAMs and came to the idea that GLMs can make predictions beyond the scope of the data used to feed the model (i.e, extrapolations), while GAMs ...
0
votes
1answer
90 views

Comparison of two measurement methods - determining measurement reliability

The aim is to compare two measurement methods. Therefore with each method two consecutive measurements were performed on 25 subjects (so overall 4 x 25 measured values) The relevant parameter is an ...