Questions tagged [two-sample]

The two-sample problem is: given samples X and Y from two distributions, test whether the two underlying distributions are the same. One of the most common classical nonparametric approach is the Kolmogorov-Smirnov test.

Filter by
Sorted by
Tagged with
1
vote
0answers
18 views

Two sample multivariate t-test?

I am wondering if there is a relatively easy way to implement a multivariate t-test or something analogous in R. I am aware of Hotelling's T-squared test, but in my case p>n and I've read that this ...
1
vote
1answer
40 views

Is Kolmogorov-Smirnov test known to fail in some special cases?

I would like to use two samples Kolmogorov-Smirnov test to check if two given samples are coming from different distributions. For that I use scipy implementation of the KS test. I have found out ...
0
votes
0answers
17 views

How to use Bayesian methods to check if the dependence variable from two dataset follow the same regression pattern?

For instance, we have two datasets with the same columns but from different years: $$\text{x1 x2 x3 y}$$ And we know that $\text{y}$ is functional related to $\text{x1 x2 and x3}$, but we don't ...
0
votes
0answers
30 views

Evaluating different methodologies for hypothesis testing

I have a campaign, with a target and holdout group, in which the target group is encouraged to make a purchase. I have the results from campaign - total revenue spent per member (for target & ...
0
votes
1answer
9 views

Calculation of critical values

Are the critical values for the two-sample t-test with unequal variances calculated the same as for the two-sample z-test?
0
votes
1answer
25 views

How to test mean differences in proportions that are dependent

I have an experimental setup where I have allowed groups consisting of about 50 animals to select between two environmental conditions. I.e. animals had the possibility to move between the two ...
2
votes
0answers
21 views

How to test whether two conditionals are equal?

I have samples from distinct joint distributions $p(x,y)$ and $q(x,y)$ and I want to determine whether $p(y|x)=q(y|x)$. Is there an existing method for doing this?
0
votes
1answer
61 views

How can you test two datasets to determine if they originate from the same generating process?

I have two feature matrices X_i and X_j, with identical features but their own n - is there a statistical test to determine if the latent data generating process behind X_i is indistinguishable from ...
1
vote
1answer
27 views

Robust two-sample test with triplicate measurements?

When testing for a difference in mean between two conditions, biologists typically use a $t$-test, and wring their hands endlessly about how to justify removing outliers. Whereas I typically use a ...
0
votes
0answers
26 views

p-value histogram of two-sample poisson tests on simulated data

I have two vectors (a and b), which are each a sample of n = 10000 from the poisson distribution with lambda = 10. ...
0
votes
0answers
94 views

Earth Movers Distance and Maximum Mean Discrepency

By Kantorovich-Rubinstein duality the Earth Movers Distance (EMD)/Wasserstein Metric is equivalent to Maximum Mean Discrepancy (MMD) correct? See here for a more thorough explanation. Why then does ...
0
votes
0answers
65 views

Kolmogorov Smirnov, CVM, AD all reject Null hypothesis - comparing two samples

I have an empirical sample (with N >10000) and I fit it to a beta-distribution. Now, as a goodness of fit, I first looked at the density function plots of the empirical and the estimated beta ...
0
votes
0answers
23 views

two-sample $T^2$ testing with $n=1$

Assuming I have two populations $X\sim\mathcal{N}(\mu_X,\Sigma)$ and $Y\sim\mathcal{N}(\mu_Y,\Sigma)$ of sizes $n_X,n_Y$, the hypotheses for equivalence of means (each of length $p$, and assuming ...
0
votes
1answer
30 views

Hypothesis Testing For Two Sample Mean Proportion

We've two samples of size n_1 and n_2 from two different normally distributed populations. We don't know their population means ...
0
votes
2answers
60 views

Which statistic test should I use in this situation?

There are 10 questions in which we are comparing peoples' opinion about two conditions, "X"and "Y". Every question was answered by about 90 people who were different than other questions' respondents. ...
0
votes
1answer
40 views

Violation of independence or still a valid test?

I have to test the the effect of two different treatments on a set of samples. On the surface, this seems an obvious option for a two sample test. The problem is that the samples being treated are ...
1
vote
0answers
26 views

Using the D-statistic from 2-sample KS-tests

this my first attempt at applying a 2-sample KS-test. I have 3 sets of data; each comprises of control and treatment values, as follows and plotted in the image: Set 1: 10,849 values each for control ...
2
votes
0answers
247 views

Testing difference in proportions with bootstrap

Summary: I want to perform a bootstrap test on the difference of proportions. I think this is equivalent to testing the difference between means using bootstrap, but I wanted to clarify some details. ...
0
votes
0answers
90 views

Confidence Interval for ratio Between Two Means

I am trying to do something similar to what it is explained in this tutorial. The difference is that I want to obtain a 95% CI for the ratio between the means of two populations. In few words, I am ...
0
votes
0answers
58 views

For a two sample t-test, what is the relationship between the p-value and the test statistic?

Suppose we have a two sample t-test where the data have unequal variances and different counts. Let $X_1, \ldots, X_{N_1}$ be the first set of data and $Y_1, \ldots, Y_{N_2}$ be the second set of data....
0
votes
1answer
91 views

Estimating Sample Size with no Standard Deviation

I am conducting a one-sided, independent two-sample t-test for a gender study and I can guesstimate my effect size from previous studies. I want 95% significance, 80% power. I am looking to calculate ...
1
vote
0answers
9 views

Looking at the plot of two sample's distribution of means which are compared cannot reveal significance? [duplicate]

In practice we sometimes used a wisdom that two overlapping CIs for sample mean distribution meant we could not reject Hull hypothesis, to get a quick answer. It looks though that this is a very raw ...
1
vote
1answer
260 views

Comparing distribution of training and test dataset

When we fit a model to a training dataset, our basic assumption is that the test dataset will also come from the same (or similar) distribution. How to check if the distribution of the training and ...
0
votes
0answers
65 views

Is there an equivalent to the Anderson-Darling test for categorical data?

I have categorical data of the following form: |Total|Pos|Neg In | 6 | 4 | 2 Out | 11 | 4 | 5 There are two categories: In/Out and Positive/Negative. All ...
0
votes
1answer
125 views

population and subsamples : mean equality?

I want to perform a two sample test, to see whether theirs means are equals. One of the sample is rather big (#60000 values) and represents the scores of a whole population. This population is ...
1
vote
0answers
25 views

Mean differences between a population and its subsamples [duplicate]

We would like to perform a statistical test to assess that a subsample of a global population has significantly a different mean than the global population. In this context, we are convinced that a ...
1
vote
0answers
433 views

AB Testing and confidence intervals

Suppose I run an AB test with the goal of increasing the number of users who purchase a product. After running the test for "long enough" I get that 5% of control users purchase, and 10% of ...
0
votes
2answers
61 views

Choosing alternate hypothesis in two sample t-test

I am trying to do two-sample T-test in Minitab where I have to compare average of one variable with another binary variable. I am having difficulty in choosing alternate hypothesis. So just for ...
1
vote
1answer
234 views

Testing if two samples have the same probability distribution

I have a sample of N integer values and I would like to check the null hypothesis $H_0 \equiv $ "the sample has a Poisson($\lambda$) distribution" This is, $p_i = P\{X=i\} = e^{-\lambda} \dfrac{\...
0
votes
0answers
51 views

two sample test for large dataset

I want to perform two sample test on large dataset (in my case, each sample size is around one billion, and each data point is single real number). Which statistical test should I choose? The null ...
4
votes
3answers
402 views

A Kernel Two Sample Test and Curse of Dimensionality

Gretton et al describes the Kernel Maximum Mean Discrepancy, a measure of distance between distributions. In order to compare two distributions, it turns out you can do much better than, say, taking ...
1
vote
0answers
57 views

Fast Two Sample Test for Bivariate Distributions

I have several thousand datasets of the same two variables (X and Y) and each dataset has at least 5,000 samples. I need to perform two-sample tests between many combinations of these datasets (total ...
-1
votes
1answer
121 views

one-sample t-test or two-sample t-test

I am interested whether or not there is a difference in a (presumably) normally distributed response variable $x$ in two conditions. There are at least two ways to tackle this: Given a set of ...
2
votes
0answers
147 views

Estimation of Covariance Matrix of Two-Sample-Two-Stage-Least-Squares Estimator

My model of interest is given by $Y_1 = X_1\beta + \epsilon_1$ with $Y_1\in\mathbb{R}^{n_1}$ , $X_1\in\mathbb{R}^{n_1}$ , $\beta\in\mathbb{R}$ and $\epsilon_1\in\mathbb{R}^{n_1}$. However, $X_1$ can ...
0
votes
1answer
5k views

How a two-tailed p-value is calculated for a two-sample t-test

I'm reading Discovering Knowledge in Data by Daniel T. Larose. In the two-sample t-test for difference in means section, there is the following example: tdata = x-bar(1) - (xbar2)/ sqrt(s^2(1)/n(1)...
1
vote
0answers
2k views

Determining when to divide alpha by 2 for significance test

I'm reviewing for my final exam and this question was asked: (help with formatting please, I'm new) The logic all makes sense to me, but their conclusions in part e and f are either wrong or really ...
1
vote
0answers
29 views

Why is the distribution of d̄ normal? (d̄ = x̄₁ - x̄₂ when both sampling distributions are normal)

I'm learning about doing inference with two samples. $d̄ = x̄₁ - x̄₂$. Sampling is independent. Let's suppose $x̄₁$ and $x̄₂$ are both normally distributed. My understanding is that this means $d̄$ ...
2
votes
3answers
139 views

Is Function of Random Samples of Two Independent Populations I.I.D.?

I want to bound the difference between two variables sampled from two different populations. When it's just one population with paired measurements a.k.a. "paired samples" (e.g. before treatment vs ...
1
vote
1answer
54 views

Comparing ordinal age group data from two samples with different scales

I have two samples of data from one population: Sample 1= 4,491 participants Sample 2 = 262 participants I want to compare age of participants in each sample but both samples have a different scale ...
3
votes
1answer
2k views

Can Friedman's test be used with two samples?

When talking about Friedman's test, it commonly comes accompanied by a whole name of "The Friedman's test for three or more correlated samples". The question is, could results be valid if I apply ...
2
votes
0answers
50 views

two samples comparison with cyclic variables

How to check if two samples are from the same distribution, when the variable is cyclic? A simple answer for non-cyclic variable is the Kolmogorov-Smirnov test. What is the equivalent for cyclic ...
1
vote
1answer
882 views

Should we apply z-test or t-test for testing the difference between two sample means?

Below is a sample question from Wayne Daniel's Biostatistics 9th ed. This is a question under the subchapter - "Sampling from normally distributed populations where population variances are known". ...
3
votes
1answer
115 views

Method to justify claim that two samples come from the same distribution

I know of ways to test "whether" two data sets come from the same distribution, in the sense that I can treat the hypothesis that they are from the same distribution as the null hypothesis. However, ...
14
votes
2answers
11k views

2 Sample Kolmogorov-Smirnov vs. Anderson-Darling vs Cramer-von-Mises

I was wondering what are the criteria to use Kolmogorov-Smirnov, Cramer-von-Mises, and Anderson-Darling when comparing 2 ECDFS. I know the mathematics of how each differ, but if I have some ECDF data, ...
0
votes
2answers
337 views

Two-sample t-test appropriate?

I have two hypotheses in which there is 1 independent variable (2 separate samples). I want to find out if the outcome of each of the 2 dependent variables is affected by the independent variable. ...
3
votes
1answer
105 views

A/B testing where success is not binary

I am a blogger testing different ad networks. The problem statement is actually quite simple: Ad network 1: 100,000 pageviews. $460 ad revenue Ad network 2: 100,000 pageviews. $428 ad revenue Can ...
1
vote
1answer
2k views

F-Statistic: Levene's Test vs. Two-Sample

I'm trying to get my head around the calculation of the critical F-Statistic for testing the equality of variances using two different tests. I have 2 groups (n=12) and (m=11). Trying to do a two ...
0
votes
0answers
29 views

How can I express $D_{m,n}$ as a distribution free statistic in terms of $d_l$

Consider 2-sample problem with m and n observations from the mutually independent, absolutely continuous populations $F_X$ and $F_Y$, respectively, define 2-sided kolmogorov-smirnov statistics $D_{m,...