Skip to main content

Questions tagged [exact-test]

Statistical tests where the sampling distribution is necessarily exactly correct & not an approximation based on assumptions &/or large sample theory.

Filter by
Sorted by
Tagged with
1 vote
1 answer
10 views

Comparing outcomes from 2 diagnostic criteria (McNemar Exact Test?)

First time poster here. I am stuck trying to figure out the correct statistical test for my research question. I am comparing two possible outcomes using the results from the same dataset. These ...
JBoud's user avatar
  • 11
0 votes
0 answers
46 views

Are there exact conditional tests for testing system of inequalities of Bernoulli means?

I have two independent Bernoulli samples: $\{X_i\}_{i=1}^{n_1}$, $\{Y_i\}_{i=1}^{n_2}$, $X_i \overset{iid}{\sim} Bern(p_1), Y_i \overset{iid}{\sim} Bern(p_2)$. I need to test: $$H_0: p_1 \ge p_2 \ge 1/...
D F's user avatar
  • 741
1 vote
1 answer
44 views

Performing "exact" G-tests in R with contingency table samples using `r2dtable` [closed]

I am interested in performing the "exact" versions of G-tests of independence in R with nonparametric table enumeration after having been told that this could help with achieving more ...
potpie's user avatar
  • 63
0 votes
4 answers
114 views

Small-sample binary logit and linear models - response to referees [closed]

Background: This cross-sectional study collected 30 thrombosis samples. We evaluated the presence or absence of MP components (dependent variable), where 24 cases had MP (coded as 1) and 6 cases did ...
zhiheng yi's user avatar
6 votes
4 answers
597 views

What is the information in an exact p-value?

Consider the following two statistical principles: 1) an exact test's $p$-value gives the exact frequency with which the observed random sample appears by chance, i.e., under a true null hypothesis; ...
virtuolie's user avatar
  • 642
2 votes
1 answer
116 views

What kind of logrank test is performed by survdiff function in R?

Information added: this question is not duplicate, neither my question nor the question in the link had been solved at all, I added more info to see if you can see the problem I was trying to ...
Estrella's user avatar
0 votes
0 answers
35 views

Performing exact tests for small samples

Let $\mathcal{X}$ be a finite set (it has, let's say, ten elements), and $p$ a probability on $\mathcal{X}$. Let $n$ be an integer, that we assume to be small (let's say, $n=10$). Given an $n$-tuple $\...
Plop's user avatar
  • 262
5 votes
0 answers
243 views

The shortest confidence interval (in the sense of expectation) for two sample t test when variances are unknown and unequal

I have had a question since graduate school. For a given significance level alpha, the test with smallest confidence interval length expectation should be optimal, because it usually also means a ...
Chang Wang's user avatar
3 votes
0 answers
80 views

Exact/Approximate Confidence Interval of Parameter Ratio from two Samples of iid Exponential

Suppose 2 independent samples $X_1,...,X_n \sim Exp(\lambda_1)$ and $Y_1,...,Y_m \sim Exp(\lambda_2)$, and are iid within samples. I am thinking about how to make an exact confidence interval for $\...
s l's user avatar
  • 87
3 votes
1 answer
816 views

Are exact logistic regression and conditional logistic regression the same?

I have seen these two terms in practice. Are they actually referring to the same method? If not, what is the main difference between the two methods? Conditional logistic regression is commonly used ...
hehe's user avatar
  • 743
0 votes
0 answers
231 views

Do poisson regression and exact poisson regression only differ in confidence interval?

The point estimates from the two approaches are the same, right? I plan to use exact Poisson regression for a study, since the events could be sparse after adjusting for a few covariates. Is there ...
hehe's user avatar
  • 743
2 votes
1 answer
187 views

What are approximate algorithms in hypothesis testing?

I am reading up 'exact test' from Wikipedia, where it says: when the result of statistical analysis is said to be an “exact test” or an “exact p-value”, it implies that the test is defined without ...
Wong's user avatar
  • 47
0 votes
1 answer
288 views

McNemar exact DP test not running in R [closed]

I am unable to run the provided example (copy/pasted from here): ...
StatsSorceress's user avatar
1 vote
2 answers
1k views

exact binomial test with power calculation that excludes the lower limit of a 95% CI

In a regulatory document I see the following statement and wonder how the sample size of 141 subjects is calculated: A sample size of 141 subjects provides 80% power to exclude a response rate of 12% ...
alittleboy's user avatar
  • 1,013
0 votes
0 answers
167 views

Which of the 3 given exact methods of calculating the confidence interval for median is better (correct)?

I want to calculate CI for the median in R. I found a number of packages and functions doing that and noticed something interesting. I think the problem can be generalized to any statistical software. ...
StatNovice's user avatar
4 votes
2 answers
330 views

Single sample t-test variance assumptions for non-normal data

So I understand that a key assumption behind the single sample t-test is that the sample mean is normally distributed. This is true when the data themselves are normally distributed, and it is ...
Zachary Miller's user avatar
9 votes
2 answers
2k views

Paired vs independent t-test for A/B test with underlying trends

There is a metric which has a natural cyclic pattern. We want to measure the effect on this metric through a A/B test. Examples: Metric is daily revenue of ice-cream trucks which is low during ...
BiGYaN's user avatar
  • 193
1 vote
1 answer
210 views

Can exact tests compare groups from case-cohort studies?

I have reviewed several helpful threads most related to my question and many thanks to the authors. The first thread suggests odds ratio is valid for cohort studies, but risk ratios or hazard ratios ...
oustella's user avatar
0 votes
1 answer
21 views

Check if multiple percentage are significant

I have 3 measurements of a very skew distribution, depending on three different technique. In general, I can do thousands of test and the result is either "fail" or "success", being the "fail" far ...
user1753235's user avatar
4 votes
1 answer
185 views

Exact distribution of number of transitions in a binary vector of length $n$

I am trying to derive an analytic solution for the exact distribution of the number of transitions in a binary vector of length $n$ conditional on the observed number of $1$'s and $0$'s and under the ...
sztal's user avatar
  • 1,191
5 votes
1 answer
3k views

Confidence interval for quantiles: distribution-free, asymptotic and assuming a normal distribution

An acquaintance of mine has been using this wrong inference formula for years: given a i.i.d. sample $\mathbf{X}={X_1,\dots,X_N}$ for a continuous RV $X$, sample mean $\bar{X}=\frac{\sum X_i}{N}$ and ...
DeltaIV's user avatar
  • 18.3k
4 votes
1 answer
167 views

Examples for exact tests which are not permutation tests

Wikipedia states (without citation): Sometimes permutation test is used as a synonym for exact test, but although all permutation tests are exact tests, not all exact tests are permutation tests. ...
Arne Jonas Warnke's user avatar
1 vote
1 answer
105 views

Number of all sample divisions in permutation test

I have a question with regard to the number of permutations in a permutation test. Assume I want to do a two group independent t-test. In the permutation variant of the t-test, I would repeatedly ...
M. Papenberg's user avatar
1 vote
1 answer
1k views

How does Normal approximation work when ties exist for Wilcoxon Signed rank test?

I ran wilcox.test() in R (outcome variable was a test score and the data was paired samples) and got a warning message saying that it cannot generate a p-value for tied values. I saw the R document ...
JNB's user avatar
  • 103
0 votes
0 answers
132 views

How to perform an exact test of Chi-square for a single-dimension data in R?

I have a data frame like this: > fr.t.r A B C D E Freq 13 3 0 0 0 and I would like to do a Chi-square test to find out if the ...
A. Caikov's user avatar
1 vote
1 answer
2k views

How IBM SPSS calculates exact p value for Pearson Chi-Square statistic?

Usually, I use Fisher's exact test p value when the sample size is small for Pearson chi-square test. Then I realized that IBM SPSS reports exact p values for all the tests (i.e. Pearson Chi-Square, ...
skorkmaz's user avatar
  • 121
0 votes
0 answers
84 views

when an exact binomial confidence interval should be used?

Is exact binomial confidence interval only appropriate for extremely rare events or extremely frequent events? If so, what are the rule of thumb for the definitions of extremely rare (<0.01?) And ...
AI2.0's user avatar
  • 133
1 vote
1 answer
485 views

Exact MCMC Logistic Regression Output

I am having trouble interpreting the output of an MCMC Logistic Regression run using R from the MCMCpack. Unfortunately I have had very little luck in finding sources on the web. I am assuming that ...
Englishman Bob's user avatar
2 votes
0 answers
223 views

LRT and $\text{Uniform}(0, \theta)$

We have one observation of variable $X$ that is distributed uniformly on $[0, \theta]$. Null hypothesis is $H_0 \colon \theta = 1$ and alternative hypothesis is $H_1 \colon \theta = 2$. What is the ...
campovski's user avatar
  • 169
6 votes
2 answers
328 views

Drawing conclusions of several inferences with the same data in one study

In a sample size of 100, we identified the existence of two attributes A and B. Our goal is to assess whether there is any ...
user81411's user avatar
  • 771
5 votes
1 answer
4k views

One-sided McNemar's test

In R, the function mcnemar.test has the following example: ...
thc's user avatar
  • 408
1 vote
0 answers
138 views

Barnard's exact tests - papers?

I am currently writing some essay for the university in which I should give a quick overview over exact methods for independence testing on contingency tables, after writing down the stuff for Fisher'...
jp399's user avatar
  • 11
1 vote
0 answers
503 views

Boschloo's exact test

I have following data in R: d1 <- matrix(c(24,2,63,11),2,2, dimnames=list(X=c("present", "absent"),Y=c("present", "absent"))) I want to test ...
user81411's user avatar
  • 771
5 votes
1 answer
2k views

z-test for one population proportion hypothesis with a small sample size

I have asked a "yes/no" question to a small group of people (n = 22). From the 22, 14 answered "yes" and 8 answered "no". I'd like to find out, with 90% confidence, if the majority of my population (> ...
Charmander's user avatar
1 vote
1 answer
661 views

Sample size determination for comparing two independent samples with regards to several continuous variables

I am interested in calculating the required sample size to perform several statistical tests on the same sample. More precisely, this is the situation: There are two groups of individuals, A and B, ...
Vicent's user avatar
  • 789
6 votes
0 answers
195 views

Exact Null Distribution with Ties

I am interested in deriving exact null distributions for small-sample test statistics with non-trivial ties. Not fundamentally continuous variables that happen to have a few repeated values, but ...
David Wright's user avatar
  • 2,261
3 votes
0 answers
772 views

pre-post testing: paired nominal data with more than 2 categories

I have two groups of students who answered a question on the pre-survey about their savings behavior, i.e. choose one answer a/b/c/e/d/f: (a) Save the same amount each week, (b) Save varied amount ...
Loon's user avatar
  • 31
1 vote
1 answer
174 views

how we do student T-Test?

In my data water is control and gibberellin is positive control. We want do find the activity of steviol against water and gibberellin. How we do student T-Test?. My data of height of plant in cm is ...
Sidra Rahman's user avatar
2 votes
0 answers
235 views

Covariates in a case control study

I am doing a retrospective analysis on a study has has cases and controls. I am looking to see if there is an association with death, using exact logistic regression. I am confused on whether I need ...
Lucy's user avatar
  • 209
3 votes
0 answers
3k views

Threshold for and Variations of Exact logistic regression

According to this UCLA tutorial exact logistic regression (elrm::elrm in R) should be prefered to the logistic regression ...
Ferdi's user avatar
  • 5,197
3 votes
1 answer
485 views

Exact multinomial goodness-of-fit test as a normality test

We have a practical real-life problem in an open source Linux related project. And I would like to hear an expert review/opinion about the way we are trying to solve this problem. It's been more than ...
Siarhei Siamashka's user avatar
5 votes
1 answer
5k views

How do you know if your confidence interval is exact or approximate?

I would like to know the rules between the two, and know how you know the differences.
das's user avatar
  • 51
1 vote
1 answer
2k views

What is the minimum sample size for an exact binomial test of goodness of fit (two-tailed)?

I am doing a two tailed exact binomial goodness of fit test, see, e.g., here. I am looking for something similar as for Chi-square tests, which, as a rule of thumb, require a frequency of 5 for each ...
You_got_it's user avatar
4 votes
1 answer
2k views

Exact test for m x n contingency table conditional (i.e. fixed by design) on one margin

I have a $m \times n$ contingency table (with $m, n > 2)$ . The experiment yields ~15% cells with expected frequencies lower 5 and also zero counts in the empirical data. The prerequisite for the ...
Mark Heckmann's user avatar
3 votes
0 answers
75 views

Test of strict difference in independent binomial probabilities

Suppose $X,Y$ are two independent binomial random variables with parameters $n_1,p_1$ and $n_2,p_2$ respectively. Suppose one wanted to test the hypothesis $p_1>p_2$. Conditional on $X+Y=s$, ...
user87290's user avatar
4 votes
1 answer
539 views

Non-parametric, repeated measures two-sample test with small sample size

My data consists in two samples relative to the expression of a given gene, with 4 replications in both samples. I am interested in testing whether the two samples come from the same population, ...
Stefano Lombardi's user avatar
2 votes
0 answers
196 views

Questions about non-standard test statistic for difference of two population proportions

Background I'm considering the number of observed 'successes' $x_1$ and $x_2$ from $n_1$ and $n_2$ independent trials respectively. These counts can be considered realisations of random variables $...
Zkk's user avatar
  • 21
9 votes
3 answers
1k views

Test to distinguish periodic from almost periodic data

Suppose I have some unknown function $f$ with domain $ℝ$, which I know to fulfill some reasonable conditions like continuity. I know the exact values of $f$ (because the data comes from a simulation) ...
Wrzlprmft's user avatar
  • 2,291
2 votes
2 answers
257 views

An exact paired perm.test is returning a p-value of 0 for large sample sizes

I'm trying to find the p-value from an exact paired permutation test. For small samples it works fine: ...
rhombidodecahedron's user avatar
5 votes
1 answer
1k views

Proper analyses for 2x2 contingency tables

So I have multiple questions about the different types of analyses that can be used on a 2x2 contingency table and when different analyses should be used. To open, I'll note that this stems from my ...
Patrick Rich's user avatar