# Questions tagged [exact-test]

Statistical tests where the sampling distribution is necessarily exactly correct & not an approximation based on assumptions &/or large sample theory.

40 questions
Filter by
Sorted by
Tagged with
47 views

### Single sample t-test variance assumptions for non-normal data

So I understand that a key assumption behind the single sample t-test is that the sample mean is normally distributed. This is true when the data themselves are normally distributed, and it is ...
105 views

### Paired vs independent t-test for A/B test with underlying trends

There is a metric which has a natural cyclic pattern. We want to measure the effect on this metric through a A/B test. Examples: Metric is daily revenue of ice-cream trucks which is low during ...
10 views

### Fixed/unfixed maginals in case/control studies on pre-existing data?

I am performing analyses on a wide variety of existing datasets (taken from GEO) with case/control status. I want to do exact tests on a 2x2 contingency table and am unsure if I should use Barnard's ...
50 views

### Can exact tests compare groups from case-cohort studies?

I have reviewed several helpful threads most related to my question and many thanks to the authors. The first thread suggests odds ratio is valid for cohort studies, but risk ratios or hazard ratios ...
12 views

### Check if multiple percentage are significant

I have 3 measurements of a very skew distribution, depending on three different technique. In general, I can do thousands of test and the result is either "fail" or "success", being the "fail" far ...
46 views

### Exact distribution of number of transitions in a binary vector of length $n$

I am trying to derive an analytic solution for the exact distribution of the number of transitions in a binary vector of length $n$ conditional on the observed number of $1$'s and $0$'s and under the ...
234 views

### Confidence interval for quantiles: distribution-free, asymptotic and assuming a normal distribution

An acquaintance of mine has been using this wrong inference formula for years: given a i.i.d. sample $\mathbf{X}={X_1,\dots,X_N}$ for a continuous RV $X$, sample mean $\bar{X}=\frac{\sum X_i}{N}$ ...
52 views

### Examples for exact tests which are not permutation tests

Wikipedia states (without citation): Sometimes permutation test is used as a synonym for exact test, but although all permutation tests are exact tests, not all exact tests are permutation tests. ...
31 views

### Number of all sample divisions in permutation test

I have a question with regard to the number of permutations in a permutation test. Assume I want to do a two group independent t-test. In the permutation variant of the t-test, I would repeatedly ...
186 views

### How does Normal approximation work when ties exist for Wilcoxon Signed rank test?

I ran wilcox.test() in R (outcome variable was a test score and the data was paired samples) and got a warning message saying that it cannot generate a p-value for tied values. I saw the R document ...
41 views

### How to perform an exact test of Chi-square for a single-dimension data in R?

I have a data frame like this: > fr.t.r A B C D E Freq 13 3 0 0 0 and I would like to do a Chi-square test to find out if the ...
592 views

### How IBM SPSS calculates exact p value for Pearson Chi-Square statistic?

Usually, I use Fisher's exact test p value when the sample size is small for Pearson chi-square test. Then I realized that IBM SPSS reports exact p values for all the tests (i.e. Pearson Chi-Square, ...
47 views

### when an exact binomial confidence interval should be used?

Is exact binomial confidence interval only appropriate for extremely rare events or extremely frequent events? If so, what are the rule of thumb for the definitions of extremely rare (<0.01?) And ...
232 views

### Exact MCMC Logistic Regression Output

I am having trouble interpreting the output of an MCMC Logistic Regression run using R from the MCMCpack. Unfortunately I have had very little luck in finding sources on the web. I am assuming that ...
82 views

### LRT and $\text{Uniform}(0, \theta)$

We have one observation of variable $X$ that is distributed uniformly on $[0, \theta]$. Null hypothesis is $H_0 \colon \theta = 1$ and alternative hypothesis is $H_1 \colon \theta = 2$. What is the ...
156 views

### Drawing conclusions of several inferences with the same data in one study

In a sample size of 100, we identified the existence of two attributes A and B. Our goal is to assess whether there is any ...
1k views

### One-sided McNemar's test

In R, the function mcnemar.test has the following example: ...
77 views

### Barnard's exact tests - papers?

I am currently writing some essay for the university in which I should give a quick overview over exact methods for independence testing on contingency tables, after writing down the stuff for Fisher'...
241 views

### Boschloo's exact test

I have following data in R: d1 <- matrix(c(24,2,63,11),2,2, dimnames=list(X=c("present", "absent"),Y=c("present", "absent"))) I want to test ...
752 views

### z-test for one population proportion hypothesis with a small sample size

I have asked a "yes/no" question to a small group of people (n = 22). From the 22, 14 answered "yes" and 8 answered "no". I'd like to find out, with 90% confidence, if the majority of my population (> ...
335 views

### Sample size determination for comparing two independent samples with regards to several continuous variables

I am interested in calculating the required sample size to perform several statistical tests on the same sample. More precisely, this is the situation: There are two groups of individuals, A and B, ...
150 views

### Exact Null Distribution with Ties

I am interested in deriving exact null distributions for small-sample test statistics with non-trivial ties. Not fundamentally continuous variables that happen to have a few repeated values, but ...
547 views

### pre-post testing: paired nominal data with more than 2 categories

I have two groups of students who answered a question on the pre-survey about their savings behavior, i.e. choose one answer a/b/c/e/d/f: (a) Save the same amount each week, (b) Save varied amount ...
164 views

### how we do student T-Test?

In my data water is control and gibberellin is positive control. We want do find the activity of steviol against water and gibberellin. How we do student T-Test?. My data of height of plant in cm is ...
152 views

### Covariates in a case control study

I am doing a retrospective analysis on a study has has cases and controls. I am looking to see if there is an association with death, using exact logistic regression. I am confused on whether I need ...
3k views

### Threshold for and Variations of Exact logistic regression

According to this UCLA tutorial exact logistic regression (elrm::elrm in R) should be prefered to the logistic regression ...
398 views

### Exact multinomial goodness-of-fit test as a normality test

We have a practical real-life problem in an open source Linux related project. And I would like to hear an expert review/opinion about the way we are trying to solve this problem. It's been more than ...
2k views

### How do you know if your confidence interval is exact or approximate?

I would like to know the rules between the two, and know how you know the differences.
1k views

### What is the minimum sample size for an exact binomial test of goodness of fit (two-tailed)?

I am doing a two tailed exact binomial goodness of fit test, see, e.g., here. I am looking for something similar as for Chi-square tests, which, as a rule of thumb, require a frequency of 5 for each ...
977 views

### Exact test for m x n contingency table conditional (i.e. fixed by design) on one margin

I have a $m \times n$ contingency table (with $m, n > 2)$ . The experiment yields ~15% cells with expected frequencies lower 5 and also zero counts in the empirical data. The prerequisite for the ...
69 views

### Test of strict difference in independent binomial probabilities

Suppose $X,Y$ are two independent binomial random variables with parameters $n_1,p_1$ and $n_2,p_2$ respectively. Suppose one wanted to test the hypothesis $p_1>p_2$. Conditional on $X+Y=s$, ...
402 views

### Non-parametric, repeated measures two-sample test with small sample size

My data consists in two samples relative to the expression of a given gene, with 4 replications in both samples. I am interested in testing whether the two samples come from the same population, ...
173 views

37 views

### exact test on proportion: confidence intervals vs p value [duplicate]

I have the following 2x2 table NO YES 9690 5 LOW 26354 39 HIGH I want to plot the proportions and ...