Questions tagged [two-sample]

The two-sample problem is: given samples X and Y from two distributions, test whether the two underlying distributions are the same. One of the most common classical nonparametric approach is the Kolmogorov-Smirnov test.

Filter by
Sorted by
Tagged with
1 vote
0 answers
17 views

How does the following term happen when using the Wald Test on two-sample binomials?

I'm having trouble understanding one of the steps of the Wald test used in two-sample binomial proportions. More specifically, when $X_1, ..., X_{n_1}$ are iid Binomial$(1, p_1$), $Y_1, ..., Y_{n_2}$ ...
user avatar
  • 11
2 votes
3 answers
56 views

Two-sample Kolmogorov-Smirnov discrete distributions

Let's say that two Census takers interview a number of couples and ask for the number of children they have. I want to compare the two resulting samples to find out if they could have come from the ...
user avatar
0 votes
0 answers
13 views

Detecting statistically significant change between multinomial samples and individual category proportions

Say I have a categorical variable with more than two categories, e.g., a person's country of origin. Assume the number of categories is some finite $k$ (e.g., not dealing with distribution of people'...
user avatar
  • 151
3 votes
0 answers
41 views

Do asymptotic statistics "solve" the Behrens-Fisher problem?

The Behrens-Fisher problem concerns comparing two means from independent (maybe multivariate) samples in a way robust to heteroskedasticity in the populations being compared. It seems that if one ...
user avatar
  • 1,063
0 votes
0 answers
20 views

Measure for similarity of two distribution ECDF with sensitivity to the tails

Currently, I am looking for a measure to quantify the overall dissimilarity or similarity of two sample distributions (possibly of different size). I would like to compare observed data with model ...
user avatar
  • 11
0 votes
0 answers
15 views

Test for equal parameters of two regression models: compare coefficients directly or check if interactions are zero?

I have two data sets and obtain regression models with coefficient vectors $\beta_1$ and $\beta_2$. I want to test $$H_0: \beta_1 = \beta_2$$ against the alternative that the two vectors are not equal....
user avatar
  • 1,063
1 vote
1 answer
36 views

Two Samples Test for Discrete Variable and Different Size Groups?

Hello there and apologies for my English and lack of statistic skills! Two different groups of participants, one comprised of children and the other of adults, run once, at different times, the same ...
user avatar
  • 111
2 votes
0 answers
38 views

Test for distribution equality

This question touches Kolmogorov-Smirnov testing, but asks actually something different. Consider independent random variables $X_1, \dots, X_n$. I want to test the following hypothesis: $$ H_0: X_i\ ...
user avatar
0 votes
0 answers
22 views

Test two samples with correlated observations within sample

I have repeated observations from individuals that have been randomly assigned to either Group 1 or Group 2. In detail, 64 subjects were randomly assigned to one of the two groups and performed a task ...
user avatar
1 vote
1 answer
32 views

Paired t-test vs two-sample t-test - animal populations

From my understanding, a paired t-test is used when samples are dependent of each other. I'm having trouble deciding whether a paired t-test should be used when comparing the average population of a ...
user avatar
0 votes
0 answers
19 views

Two Sample test for two samples with different distribution

I have independent random samples of $R^d, d>1$ $X=\{X_1,..X_n\}$ and $Y=\{Y_1,.. Y_n\}$ with respective distributions $F_1$ and $F_2$. In a regular two sample problem we test $H_0:F_1=F_2$ vs $H_1:...
user avatar
1 vote
0 answers
36 views

Testing difference of medians and stochastic ordering using paired samples

I have paired samples from skewed distributions with unequal variance (before and after treatment), and would like to test for difference of means difference of medians stochastic ordering of treated ...
user avatar
  • 41
1 vote
0 answers
59 views

With 2-sample bootstrap testing, why pool treatment with control samples?

I have data from a randomized experiment. Members of the treatment group received a treatment, members of the control group did not. I want to test differences in distributions of ordinal outcomes ...
user avatar
  • 11
0 votes
2 answers
67 views

Two-sample test with bootstrap and different sizes

I have two independent i.i.d., respectively $Y_1,\dots,Y_{n_Y} \sim F_Y$ of size $n_Y$ and $X_1,\dots,X_{n_X} \sim G_X$ of size $n_X$. I would like to test: $$ H_0:\mu_X = \mu_Y $$ versus any ...
user avatar
  • 67
1 vote
1 answer
85 views

How to perform F-test in R for checking equality-of-variance (homoscedasticity/heteroscedasticity) if raw data is not given?

In a typical two-sample problem scenario, I possess the sample mean, variance, and size, but not the raw data for samples drawn from two populations. I wish to perform the F-test to check equality of ...
user avatar
1 vote
2 answers
49 views

What is the theoretical justification of testing for equal mean/distribution via resampling?

If we have two data sets $X_1,\ldots,X_m$ and $Y_1,\ldots,Y_n$, each i.i.d., and wanted to determine whether $\mathbb{E}[X_1] = \mathbb{E}[Y_1]$ or not using $\bar{X}_m - \bar{Y}_n = \hat{\Delta}_{m,n}...
user avatar
  • 1,063
0 votes
0 answers
28 views

Aggregating individual ks 2 samp results?

Lets say I have two servers each reporting an individual time series of cpu usage, ts_a and ts_b. I can average these two CPU usage metrics together to get an averaged or "composite" time ...
user avatar
1 vote
2 answers
320 views

Bayesian : Comparing means of two posterior samples/ Help a Frequentist Out

UPDATE Thanks for the many thoughtful responses and questions! I've made edits here to clarify further. and also respond to each respondent individually. Original Post I have two sets of posterior ...
user avatar
0 votes
0 answers
35 views

Comparing differences of two subsamples in two samples

I have two samples from different populations: say Children versus Adult and two subgroups within each sample (say ...
user avatar
  • 1
0 votes
0 answers
41 views

Two sample similarity analysis for non-independent daily time series data

What two sample or other relevant statistical tests can be performed in R between two samples which are independent of each other, but the observations within each ...
user avatar
2 votes
1 answer
60 views

Doing a t-test on a normally distributed difference of two non normal distributions

I have two populations of donations (segment 2 people were subject to a different form): ...
user avatar
3 votes
2 answers
310 views

What difference between Mann-whintey U-test and Kolmogorov-Smirnov test on truncated log normal distributions?

I have two populations who have been exposed to two different websites that should bring them to donations: one with a progress bar that pushes them to give (B, segment 2) and the other not (A, ...
user avatar
3 votes
3 answers
117 views

T-test states difference of donation is significant when Z-test claims not, what method to use?

I have two populations who have been exposed to two different websites that should bring them to donations: one with a progress bar that pushes them to give (B, segment 2) and the other not (A, ...
user avatar
3 votes
1 answer
135 views

Why is the two-sample test giving me inconsistent results?

I am applying a two-sample t-test to determine whether we have software regressions on latency measurements. Procedure Run the test for build b1 and gather 60 latency measurements. Run the test for ...
user avatar
  • 157
5 votes
1 answer
51 views

Why am I observing non-uniformly distributed (negatively skewed) p-values for two-sample tests of mixture distributions when the null is true?

I am interested in generating Gaussian mixture distributions as the null distributions for a series of two-sample test simulations. It is a well established fact that p-values follow a uniform ...
user avatar
0 votes
1 answer
35 views

t-statistic varies widely when used for bootstrap sampling process

I have two samples of size 20 and I want to see if the difference between the geometric means is significant using two-sample t-statistic and bootstrap sampling. Here is what I'm doing ...
user avatar
  • 25
3 votes
2 answers
79 views

Testing for equal proportions when sample sizes are very small

Suppose I observe binary data for two samples (hopefully the notation below is obvious) and I wish to test the hypotheses: $$H_0: p_1 = p_2$$ $$H_A: p_1 \neq p_2$$ I know there is a $z$-test for doing ...
user avatar
  • 1,063
5 votes
2 answers
8k views

what is the difference between a two-sample t-test and a paired t-test

While I was glancing at hypothesis tests, I saw paired and two-sample t-test but couldn't understand the difference. For the explanation of these two tests, I saw the following sentence " Two-...
user avatar
2 votes
1 answer
152 views

Can I simulate a two-sample K-S test when I can't perform a one-sample test because a distribution parameter is unknown?

Let's say I have a signal which I want to test for normality. I know that the mean from the theoretical distribution is zero, but it has an undetermined (unknown) variance. If I knew mean and variance ...
user avatar
  • 23
3 votes
0 answers
497 views

R function for weighted two-sample t-test **with Welch-adjusted t statistic**?

I'm conducting a hypothesis test for the difference between two groups. The file dat1 contains all observations of measure for ...
user avatar
  • 135
1 vote
0 answers
36 views

Determine distribution of a new test statistic

Question from my course project: Suppose we are conducting a two-arm trial. For the control group, there are $n_c$ patients, and their outcome is denoted by X, which has distribution $N(\mu_{c},\sigma^...
user avatar
0 votes
1 answer
84 views

Two-sample test + paired/independent samples

This might be an easy question but I got lost in my thinking so thank you for clarifying that for me. So I have some troubles with understanding the ideas of two-sample tests in addition to paired and ...
user avatar
2 votes
3 answers
79 views

The mean of natural log of some data

I have the following data in natural log: N = [0.929, -1.745, 1.677, 0.701, 0.128] O = [2.233, -2.513, 1.204, 1.938, 2.533] I want to calculate the mean ratio of ...
user avatar
1 vote
1 answer
73 views

Confused about Two Sample Proportion Tests - am I performing this correctly?

I am dealing with an advertising dataset that includes hundreds of advertisements, how many clicks each ad receives, whether the amount of clicks received is considered "high" or "low&...
user avatar
  • 11
1 vote
0 answers
71 views

Two-sample test when samples are not completely independent?

Let's say I measure the height of 100 individuals every week in both Illinois and Texas while allowing for the same individual to be measured multiple times in the following weeks. This way, my ...
user avatar
0 votes
2 answers
281 views

Performing a two sample t-test for equal means on subset of a dataset

If I have a dataset that, of among other variables, contains the gender and wealth for a sample of a population with a year variable that represents the year the data about the individual was ...
user avatar
2 votes
0 answers
47 views

test whether two samples are from the same high dimensional multinomial distribution

I have two high dimensional vectors (~150 dims), assumed to be sampled from multinomial distributions. How can I test whether the two are from the same multinomial distribution or different ones? (...
user avatar
-1 votes
1 answer
163 views

Kolmogorov-Smirnov two-sample test p-value evaluation

I'm using python's scpipy implementation of the Kolmogorov-Smirnov two-sample test (ks_2samp). Q1: Why are my p-values in the examples below so low, specifically on the first example? Q2: Can I ...
user avatar
  • 13
0 votes
1 answer
102 views

two-sample Kolmogorov-Smirnov test suggest that sample with the same data generating process from different populations

I have two different samples I need to test if they are drawn from the same population distribution. The two samples are of the volatility of an asset that I simulated. Since the data is simulated I ...
user avatar
1 vote
1 answer
208 views

Two-sample Kolmogorov-Smirnov, Anderson-Darling, and after?

I am trying to investigate the differences between two datasets. Each one is composed of 20 000 elements obtained from a Monte-Carlo method whose implementation slightly differs in both cases. I'd ...
user avatar
  • 121
2 votes
0 answers
27 views

Likelihood as a test statistic in a hypothesis test

Suppose I have two samples $S_1,S_2$ of categorical data, and I'd like to design a hypothesis test to check the null hypothesis that they are both iid samples from the same underlying distribution. ...
user avatar
  • 6,003
0 votes
0 answers
819 views

Compare distributions using Maximum Mean Discrepancy (MMD)

I use MMD distance to run a permutation test and decide whether two sample distributions come from the same distribution or not. For the MMD, I use a gaussian kernel, the bandwidth of which I select ...
user avatar
5 votes
3 answers
9k views

confidence interval for 2-sample t test with scipy

...
user avatar
1 vote
0 answers
17 views

Two-group "ANCOVA" with Theil-Sen?

("ANCOVA" might typically refer to using at least three groups. In what I do, I am always comparing two groups, and that is what I mean throughout this question.) I know how to do ANCOVA via ...
user avatar
  • 32.4k
2 votes
1 answer
2k views

How to interpret results of two-sample, one-tailed t-test in Scipy

I want to perform a two-sample, one-tailed t-test to compare two means. For the specific problem I am looking, I want the comparison to only be in one direction. I would like the null hypothesis to be ...
user avatar
1 vote
0 answers
18 views

Comparing Adult Weights over time - Should I use 1 or 2 sample test?

Background: Many years ago, we were looking to calculate adult weights belonging to a certain population. At the time, it was assumed that the population mean was 77kg. After a survey, we got the ...
user avatar
3 votes
3 answers
938 views

Two one-sided hypothesis tests instead of a two-sided test?

In hypothesis testing, the guidance is to use a one-sided test (alternative "greater" or "lesser") if we don't care about errors in one of the directions. If we do care about errors in both directions ...
user avatar
  • 2,280
3 votes
0 answers
49 views

Two-sample KS test null distribution, reference request

I follow, more or less, the derivation of the KS test statistics's distribution that is given on Wikipedia. The following section on the two-sample test also makes sense if all I want to do is reject ...
user avatar
  • 32.4k
0 votes
0 answers
86 views

Two-sample t test with before and after experiment data

I have data with the following columns: Person id, achievement before treatment, achievement after treatment (timewise) Each person is put into a control or treatment group. Because the data is ...
user avatar
1 vote
1 answer
251 views

Using one-sample Kolmogorov-Smirnov test impementation for comparing two samples with purely discrete CDF

In one-sample Kolmogorov-Smirnov test one tests whether sample $X = (X_1,\dots,X_n)$ come from theoretical distribution by computing $D_n = \sup_{x} |F_n(x) - F(x)|$, where $F_n(x)$ denotes the ...
user avatar