Skip to main content

Questions tagged [two-sample]

The two-sample problem is: given samples X and Y from two distributions, test whether the two underlying distributions are the same. One of the most common classical nonparametric approach is the Kolmogorov-Smirnov test.

Filter by
Sorted by
Tagged with
0 votes
0 answers
33 views

Is bootstrapping inherently Frequentist? If so, how do we do a Bayesian non-parametric two-sample test?

I normally use frequentist statistics but I now want to use Bayesian statistics as I want to carry out a two-sample (randomised control trial) test that includes prior information. I have an existing ...
Amorphia's user avatar
  • 913
1 vote
0 answers
43 views

Deriving Sample Size for two samples t test

I am trying to figure out what sample size recommendations to make in this case when the samples have different sizes. Since their sizes are different the variance of each will be different. Suppose ...
Harry Lofi's user avatar
1 vote
1 answer
23 views

Two-sample test of difference in probability mass at the extremes of the empirical distributions

I am running an experiment that will generate a dependent variable (DV) in two treatments, T1 and T2. One of the hypotheses I want to test is whether the distribution of the DV in T1 has more mass at ...
hangingprawns's user avatar
1 vote
1 answer
63 views

How to perform a 2 sample Kolmogorov-Smirnov test controlling for a type II error instead of type 1

I have four completely different sets of two samples and want to check for each set if both samples come from the same distribution, so four Kolmogorov-Smirnov tests total. If it's going to be ...
Frank's user avatar
  • 21
0 votes
0 answers
46 views

How to Run a Standard Two-Sample t-test when given delta and not the two means

I am trying to find the p-value to test a null hypothesis using a Standard Two-Sample t-Test. Usually I would use ...
user399677's user avatar
0 votes
0 answers
20 views

Validity of Bootstrap Inference for Bounds

I am facing the following problem: I have access to iid $X_1,\dots ,X_{N_X}$ and $Y_1, \dots, Y_{N_Y}$ from $F_X$ and $F_Y$, respectively, where $X$ stochastically dominates $Y$. My goal is to conduct ...
Tommo's user avatar
  • 1
0 votes
0 answers
11 views

Two-sample statistical significance test taking into account group membership (proportions) and numeric comparison

Say I have two samples of observations, S1 and S2, which can be of unequal sizes. For each observation, I have two variables: G (a categorical group assignment, say it is binary with values A/B), and ...
Sam A.'s user avatar
  • 171
1 vote
1 answer
78 views

Two sample proportions: How to test the hypothesis that the difference in proportions is less than a specified threshold?

Say I have treatment and control, perfectly randomized. I have 70000 conversions for treatment in 80000 users. I have 70100 converions for control in 800100 users. My null hypothesis is that there is ...
Estimate the estimators's user avatar
0 votes
1 answer
25 views

Is two-sample Kolmogorov test working correctly?

I have two one-dimensional samples that I'm trying to quantifiably distinguish (or deny such distinction). I.e. the null-hypothesis is that they come from the same population (distribution?). The ...
Eugene B.'s user avatar
1 vote
0 answers
86 views

Comparing means in Box-Cox transformed data

Deeply sorry as this wasn't really covered in my statistics classes. I am current comparing datasets for people divorcing vs. dissolving their marriages. I have two variables, length of marriage, and ...
Sophie's user avatar
  • 11
1 vote
1 answer
94 views

Kolmogorov Smirnov Test Consistency

I was reading that the Kolmogorov Smirnov 2 sample test is consistent, that is Probability of rejection under $H_1$ is 1 for sample size going to infinity. Say we have 2 random variables X and Y. K-S ...
Andrew741's user avatar
1 vote
1 answer
268 views

When transforming a dataset using the square root (or any other procedure), does the standard deviation of the dataset get transformed too?

Taking an introductory statistics course, and I am confused about the impact of a transformation on the df for a two-sample t-test. The df is dependent on n of each sample and SD of each sample, but I'...
Llove23's user avatar
  • 11
0 votes
1 answer
72 views

Compare two samples when each observation is a pair of dependent values

At an airport there is a company that offers a specific service for travellers (there are no competitors offering the same service at the location). They performed an experiment - they temporarily ...
mcihak's user avatar
  • 157
1 vote
0 answers
16 views

Permutation Testing Using only Source Samples

Short Background on Permutation Testing Suppose I have two sets of samples $P$ and $Q$ drawn iid from distributions $\mathcal{P}$ and $\mathcal{Q}$ over $X$. I also have access to a test function $T: ...
user2757771's user avatar
2 votes
1 answer
177 views

How to compare multiple Two-Sample Hotelling's T-squared tests performed on different sample sizes

It's my first question, so please be patient with me. I'm working on the comparison of multiple datasets of different lengths (60 to ~10000). They share a set of 9 features which I would like to use ...
rbrsmn's user avatar
  • 21
0 votes
1 answer
113 views

two-sample proportions hypothesis test: 10 successes and failures rule

I asked this question on mathematics stack exchange, and reposting here as it did not get resolved. Suppose the sample sizes and number of successes are ($n_1, y_1$) and ($n_2, y_2$), for the two ...
creet's user avatar
  • 3
2 votes
0 answers
62 views

How does the following term happen when using the Wald Test on two-sample binomials?

I'm having trouble understanding one of the steps of the Wald test used in two-sample binomial proportions. More specifically, when $X_1, ..., X_{n_1}$ are iid Binomial$(1, p_1$), $Y_1, ..., Y_{n_2}$ ...
Chris's user avatar
  • 21
3 votes
3 answers
1k views

Two-sample Kolmogorov-Smirnov discrete distributions

Let's say that two Census takers interview a number of couples and ask for the number of children they have. I want to compare the two resulting samples to find out if they could have come from the ...
Rafael L's user avatar
4 votes
0 answers
122 views

Do asymptotic statistics "solve" the Behrens-Fisher problem?

The Behrens-Fisher problem concerns comparing two means from independent (maybe multivariate) samples in a way robust to heteroskedasticity in the populations being compared. It seems that if one ...
cgmil's user avatar
  • 1,373
1 vote
1 answer
259 views

Two Samples Test for Discrete Variable and Different Size Groups?

Hello there and apologies for my English and lack of statistic skills! Two different groups of participants, one comprised of children and the other of adults, run once, at different times, the same ...
Avi's user avatar
  • 111
2 votes
0 answers
92 views

Test for distribution equality

This question touches Kolmogorov-Smirnov testing, but asks actually something different. Consider independent random variables $X_1, \dots, X_n$. I want to test the following hypothesis: $$ H_0: X_i\ ...
Albert Paradek's user avatar
0 votes
0 answers
28 views

Test two samples with correlated observations within sample

I have repeated observations from individuals that have been randomly assigned to either Group 1 or Group 2. In detail, 64 subjects were randomly assigned to one of the two groups and performed a task ...
Researcher's user avatar
1 vote
1 answer
165 views

Paired t-test vs two-sample t-test - animal populations

From my understanding, a paired t-test is used when samples are dependent of each other. I'm having trouble deciding whether a paired t-test should be used when comparing the average population of a ...
user avatar
0 votes
0 answers
25 views

Two Sample test for two samples with different distribution

I have independent random samples of $R^d, d>1$ $X=\{X_1,..X_n\}$ and $Y=\{Y_1,.. Y_n\}$ with respective distributions $F_1$ and $F_2$. In a regular two sample problem we test $H_0:F_1=F_2$ vs $H_1:...
michael's user avatar
1 vote
0 answers
47 views

Testing difference of medians and stochastic ordering using paired samples

I have paired samples from skewed distributions with unequal variance (before and after treatment), and would like to test for difference of means difference of medians stochastic ordering of treated ...
alfba's user avatar
  • 41
2 votes
0 answers
187 views

With 2-sample bootstrap testing, why pool treatment with control samples?

I have data from a randomized experiment. Members of the treatment group received a treatment, members of the control group did not. I want to test differences in distributions of ordinal outcomes ...
Scorocs's user avatar
  • 21
2 votes
2 answers
775 views

Two-sample test with bootstrap and different sizes

I have two independent i.i.d., respectively $Y_1,\dots,Y_{n_Y} \sim F_Y$ of size $n_Y$ and $X_1,\dots,X_{n_X} \sim G_X$ of size $n_X$. I would like to test: $$ H_0:\mu_X = \mu_Y $$ versus any ...
Barbab's user avatar
  • 333
1 vote
1 answer
633 views

How to perform F-test in R for checking equality-of-variance (homoscedasticity/heteroscedasticity) if raw data is not given?

In a typical two-sample problem scenario, I possess the sample mean, variance, and size, but not the raw data for samples drawn from two populations. I wish to perform the F-test to check equality of ...
Lycanthropeus's user avatar
1 vote
2 answers
82 views

What is the theoretical justification of testing for equal mean/distribution via resampling?

If we have two data sets $X_1,\ldots,X_m$ and $Y_1,\ldots,Y_n$, each i.i.d., and wanted to determine whether $\mathbb{E}[X_1] = \mathbb{E}[Y_1]$ or not using $\bar{X}_m - \bar{Y}_n = \hat{\Delta}_{m,n}...
cgmil's user avatar
  • 1,373
1 vote
2 answers
2k views

Bayesian : Comparing means of two posterior samples/ Help a Frequentist Out

UPDATE Thanks for the many thoughtful responses and questions! I've made edits here to clarify further. and also respond to each respondent individually. Original Post I have two sets of posterior ...
pythOnometrist's user avatar
1 vote
0 answers
65 views

Two sample similarity analysis for non-independent daily time series data

What two sample or other relevant statistical tests can be performed in R between two samples which are independent of each other, but the observations within each ...
Ed_Gravy's user avatar
  • 111
2 votes
1 answer
105 views

Doing a t-test on a normally distributed difference of two non normal distributions

I have two populations of donations (segment 2 people were subject to a different form): ...
Revolucion for Monica's user avatar
6 votes
2 answers
3k views

What difference between Mann-whintey U-test and Kolmogorov-Smirnov test on truncated log normal distributions?

I have two populations who have been exposed to two different websites that should bring them to donations: one with a progress bar that pushes them to give (B, segment 2) and the other not (A, ...
Revolucion for Monica's user avatar
3 votes
3 answers
182 views

T-test states difference of donation is significant when Z-test claims not, what method to use?

I have two populations who have been exposed to two different websites that should bring them to donations: one with a progress bar that pushes them to give (B, segment 2) and the other not (A, ...
Revolucion for Monica's user avatar
5 votes
1 answer
253 views

Why is the two-sample test giving me inconsistent results?

I am applying a two-sample t-test to determine whether we have software regressions on latency measurements. Procedure Run the test for build b1 and gather 60 latency measurements. Run the test for ...
Klik's user avatar
  • 187
5 votes
1 answer
85 views

Why am I observing non-uniformly distributed (negatively skewed) p-values for two-sample tests of mixture distributions when the null is true?

I am interested in generating Gaussian mixture distributions as the null distributions for a series of two-sample test simulations. It is a well established fact that p-values follow a uniform ...
computationalstatistician's user avatar
0 votes
1 answer
230 views

t-statistic varies widely when used for bootstrap sampling process

I have two samples of size 20 and I want to see if the difference between the geometric means is significant using two-sample t-statistic and bootstrap sampling. Here is what I'm doing ...
kyc12's user avatar
  • 145
4 votes
2 answers
2k views

Testing for equal proportions when sample sizes are very small

Suppose I observe binary data for two samples (hopefully the notation below is obvious) and I wish to test the hypotheses: $$H_0: p_1 = p_2$$ $$H_A: p_1 \neq p_2$$ I know there is a $z$-test for doing ...
cgmil's user avatar
  • 1,373
8 votes
2 answers
36k views

what is the difference between a two-sample t-test and a paired t-test

While I was glancing at hypothesis tests, I saw paired and two-sample t-test but couldn't understand the difference. For the explanation of these two tests, I saw the following sentence " Two-...
Atilla Colak's user avatar
3 votes
1 answer
312 views

Can I simulate a two-sample K-S test when I can't perform a one-sample test because a distribution parameter is unknown?

Let's say I have a signal which I want to test for normality. I know that the mean from the theoretical distribution is zero, but it has an undetermined (unknown) variance. If I knew mean and variance ...
Phxuibs's user avatar
  • 33
5 votes
1 answer
2k views

R function for weighted two-sample t-test **with Welch-adjusted t statistic**?

I'm conducting a hypothesis test for the difference between two groups. The file dat1 contains all observations of measure for ...
JmQ's user avatar
  • 129
1 vote
0 answers
49 views

Determine distribution of a new test statistic

Question from my course project: Suppose we are conducting a two-arm trial. For the control group, there are $n_c$ patients, and their outcome is denoted by X, which has distribution $N(\mu_{c},\sigma^...
Schnappiii's user avatar
0 votes
1 answer
552 views

Two-sample test + paired/independent samples

This might be an easy question but I got lost in my thinking so thank you for clarifying that for me. So I have some troubles with understanding the ideas of two-sample tests in addition to paired and ...
pelican456's user avatar
2 votes
3 answers
170 views

The mean of natural log of some data

I have the following data in natural log: N = [0.929, -1.745, 1.677, 0.701, 0.128] O = [2.233, -2.513, 1.204, 1.938, 2.533] I want to calculate the mean ratio of ...
Ramy Maher's user avatar
1 vote
1 answer
211 views

Confused about Two Sample Proportion Tests - am I performing this correctly?

I am dealing with an advertising dataset that includes hundreds of advertisements, how many clicks each ad receives, whether the amount of clicks received is considered "high" or "low&...
Bert's user avatar
  • 11
1 vote
0 answers
180 views

Two-sample test when samples are not completely independent?

Let's say I measure the height of 100 individuals every week in both Illinois and Texas while allowing for the same individual to be measured multiple times in the following weeks. This way, my ...
Voldemort's user avatar
1 vote
2 answers
823 views

Performing a two sample t-test for equal means on subset of a dataset

If I have a dataset that, of among other variables, contains the gender and wealth for a sample of a population with a year variable that represents the year the data about the individual was ...
Brandon's user avatar
  • 11
2 votes
0 answers
114 views

test whether two samples are from the same high dimensional multinomial distribution

I have two high dimensional vectors (~150 dims), assumed to be sampled from multinomial distributions. How can I test whether the two are from the same multinomial distribution or different ones? (...
user294869's user avatar
-1 votes
1 answer
326 views

Kolmogorov-Smirnov two-sample test p-value evaluation

I'm using python's scpipy implementation of the Kolmogorov-Smirnov two-sample test (ks_2samp). Q1: Why are my p-values in the examples below so low, specifically on the first example? Q2: Can I ...
beerzy's user avatar
  • 33
0 votes
1 answer
215 views

two-sample Kolmogorov-Smirnov test suggest that sample with the same data generating process from different populations

I have two different samples I need to test if they are drawn from the same population distribution. The two samples are of the volatility of an asset that I simulated. Since the data is simulated I ...
BenAlbert's user avatar