Questions tagged [hypothesis-testing]
Hypothesis testing assesses whether data are inconsistent with a given hypothesis (usually a null hypothesis of no effect).
10,986 questions
1
vote
0
answers
36
views
Recipes for combining multiple null-hypothesis tests
I have two questions about hypothesis testing:
Is there one or multiple generic recipes for combining tests together? Some procedure that would produce a new better test $T_\star$ from two tests $T_1$...
- 3,303
1
vote
0
answers
37
views
What type of test should I use for paired categorical data?
Somewhat similar questions have been asked here and here, but none of the other answers are really applicable in my case.
I have a group of people, where in Year 1 everybody is sick. Here I have only ...
- 11
2
votes
1
answer
45
views
The F-test and the t-test reject the null hypothesis while the KS test and the Chi-Squared test do not
This is a follow-up question to How can the F-test reject the null hypothesis while the KS test does not?.
Let's consider I perform the following tests:
Two-sample Kolmogorov-Smirnov test (KS-test),
...
- 444
4
votes
4
answers
95
views
Statistical Inference on Samples vs Populations
I am looking at this question Statistical inference when the sample "is" the population where the following analysis is given:
User 1: "Treat the population data as a sample and assume ...
user420944
1
vote
3
answers
78
views
How to show that many functions (a hundred, a thousand) have the same shape an distribution of values over an interval?
I have functions that on iterval [0,1] all seem to look like this:
i.e. they have a zero around 0.4 +ve derivative from zero to 0.4 and around zero or slightly negative derivative up to 1.
I plan to ...
- 13
0
votes
0
answers
12
views
understanding fisheries model output tag and release
I am reviewing (for myself) a publication that has analysed data from an experiment to see what effect underwater seismic has on catchability of lobsters. de Lestang et al (2024) Fisheries Research ...
1
vote
0
answers
54
views
Hypothesis Test on Whole Population? [closed]
Although this question was discussed here Do we need hypothesis testing when we have all the population? , I wanted to get some second opinions on this topic.
I have the following set up for some ...
user420944
3
votes
3
answers
105
views
Can hypothesis testing be done on two points? [closed]
I have the following set up for some island (data is available for each individual islander and the entire population is available):
Year 2000 :
Total Adult Population: 260,000;
Number of Healthy ...
user420944
2
votes
1
answer
47
views
How does one develop an intuitive understanding of statistical methods and the ability to "know" which test to use, why, and how to interpret results [closed]
This is not a mathematical or coding question. Apologies if this is not the forum for this and please let me know if I need to ask this somewhere else.
I have a masters degree in data science and I ...
2
votes
1
answer
50
views
ANOVA finding a main effect prevents inflation of Type I error rate
In the (free) book Understanding Statistics and Experimental Design, section 6.3, they say
'...ANOVA offers a second trick. If we rejected the null hypothesis, it is appropriate to compare pairs of ...
- 53
0
votes
1
answer
29
views
What criterion to use to compare multiple correlations of binary variables?
I have $N$ definitions of certain properties of countries (for example, if the country is "democratic", "totalitarian" etc.), and want to test how consistently different people ...
- 101
0
votes
0
answers
27
views
Issues with Detecting Negative Change in AR(1) Model Using CPAT Library
I'm currently working on evaluating the empirical power of the CUSUM, Rényi CUSUM, and Erdős-Darling tests for detecting change points in AR(1) models, specifically focusing on changes in the mean. ...
- 11
0
votes
0
answers
32
views
Appropriate way to correct for multiple comparisons in this case
I am trying to compare the performance of an optimisation algorithm on different objective functions. The functions can vary in their form (4 different types) and dimension. I would like to first ...
- 13
1
vote
1
answer
66
views
Power of One Sided t-test
Let $X_1, \ldots, X_n$ be a sample from $N(\mu, \sigma^2)$ for unknown $\mu \in \mathbb{R}$ and unknown $\sigma > 0$. Fix $\mu_0 \in \mathbb{R}$. The one-sided hypothesis is $H_0: \mu \leqslant \...
- 639
6
votes
1
answer
435
views
Related low p-values that do not meet statistically significant thresholds
If I were to have multiple related measures, each with "low" p-values that do not quite meet the threshold of significance, could the fact that multiple measures produce a general trend with ...
1
vote
2
answers
50
views
Looking for a statistical test for significance of association between a treatment and non-mutually exclusive categories
I am looking for a statistical test which will give a p value for the association between a treatment and two potentially co-occurring labels:
I have two categories of cells: state A and state B.
A ...
- 11
0
votes
0
answers
34
views
How the combined test under Fisher's method is generally stronger than individual tests, mathematically?
I have gone through the definition of Fisher's method in several materials but it is unclear how Fisher's method is stronger than individual tests. I guess it is not always the case, i.e., there are ...
1
vote
2
answers
66
views
Test whether my new method's type I error rate correctly matches chosen alpha
I am testing two different methods for hypothesis testing, bootstrapped likelihood ratio tests, and likelihood ratio tests assuming an asymptotic distribution. I would like to see if the number of ...
- 425
0
votes
0
answers
20
views
Testing equality of multiple proportions in mixing problems
There are two lakes A (43000 cubic feet) and B (30000 cubic feet). A factory dumps waste matter into both lakes. A measurement yields three types of pollutants a, b, c.
The percentage of the lakes ...
1
vote
0
answers
30
views
Paired Non-parametric test with more than 3 groups but just two timepoints
I may not have found the right term to search for, so I am asking what might be a straightforward question:
Assume we have an experiment with paired non-normal distributed data, called "Pre" ...
0
votes
1
answer
44
views
ML V REML for Hypothesis Testing
I am curious about doing hypothesis testing with REML values via a likelihood ratio test, or whatever test would make sense for REML values, but these two highly upvoted answers to existing cross ...
- 425
1
vote
2
answers
58
views
Impact of sample size on metric lift
Assume we ran three randomized user split AB experiments exp_1, exp_2, exp_3 one after the other. All three had the exact same treatments & control.
Exp_1 was ran for 2 weeks during 06/01-06/14 . ...
- 43
1
vote
0
answers
38
views
Combining Data from Multiple Participants for Statistical Analysis of a Facial Expression Memory Task
Context of the Query: I am working on a project involving a memory task for facial expressions, consisting of two phases:
Encoding Phase: In this phase, participants are exposed to 8 virtual ...
0
votes
0
answers
57
views
Equivalent of one-tailed Fisher's exact test for enrichment of the diagonal in 3x3 contingency table?
I have 2 ordinal variables A and B with 3 levels each (e.g. "down", "no change", and "up"). I want to know whether the up/down changes indicated by A and B are consistent ...
- 383
0
votes
0
answers
21
views
Reporting significance of fixed effects in clmms - when to use ANODE tables?
I am working on some mixed effects models for ordinal outcomes using clmm. I have settled on my final model, and now want to present the findings. On reading around, I see the use of ANODE tables both ...
- 1
2
votes
0
answers
33
views
Problem regarding False discovery rate correction for multiple comparison
I am conducting research to find association of a gene with a disease. It's a case-control study. I have divided the cases into groups A and B based on genotype and controls into groups C and D. I ...
4
votes
4
answers
192
views
Non-parametric one-sample mean test for a bounded variable (based on Chebyshev's inequality?)
The problem
I have $x_1, \ldots , x_n$ i.i.d. draws from r.v. $X$ such that $0 \leq X \leq 1$, but I can't make any other assumptions about the distribution.
I want to test the null hypothesis that $E(...
- 3,144
0
votes
0
answers
24
views
What is the appropriate statistical test in this scenario?
I work with a particular genus of plants where there have been many instances where what was once considered a species is now considered to be a synonym of another species. I am interested in ...
- 11
2
votes
1
answer
69
views
How can I show statistically that one of my replicates is likely contaminated?
I have a dataset that looks like the below: five replicate samples, each of which is composed of 4 different fractions that sum to 100%. The fifth sample clearly looks visually distinct from the other ...
- 288
4
votes
2
answers
169
views
Are there alternative statistics to a p-value in NHST?
Sometimes, when the logic of p-value is explained and motivated, it happens in the following way:
The concept of a random sample is introduced, together with the need to consider the possibility that ...
- 777
2
votes
2
answers
215
views
Is it possible for some p-values to be impossible? (because statistic generated by parametric bootstrap is mostly the same value.)
I am using a parametric bootstrap/monte carlo hypothesis testing method to generate the null distribution of the log likelihood ratio statistic. However, I am worried I might be doing it wrong ...
- 425
0
votes
0
answers
32
views
Apply the Breusch-Pagan to uncover the source of variance in the data
For my research I am trying to uncover if the variance of my variable of interest is correlated with one of my independent variables. Let $ x $ be the independent variable of interest and $ m $ be the ...
0
votes
0
answers
29
views
Difference in means and variances of dependent samples
I am performing a simulation study where different values of an exposure X are repeatedly assigned to the same individual (200x in a sample of >10000 people). The way that assignment is done is ...
- 21
3
votes
1
answer
171
views
Why do we compute the standard deviation of the proportion using the value assumed under the null hypothesis?
In type 2 error computation, why do we compute the standard deviation using the value assumed under the null hypothesis even if we provide the alternative hypothesis as possible true value?
If in null ...
- 33
4
votes
2
answers
365
views
confidence interval and rejection
Assume I have a data sample $X_{1}, \dots X_{n}$ from a normal distribution. Then, we consider the following hypothesis
$$
H_0: \quad \mu = \mu_{0}
$$
and alternative
$$
H_1: \quad \mu \neq \mu_{0}
$$
...
- 668
1
vote
2
answers
27
views
What is correct test to investigate difference between two conditions with repeated measures per subject?
I am having trouble deciding what statistical test I should use for my data. I have 5 subjects who rated attractiveness of 20 objects and 20 sounds of the objects on a scale of 1-5. I would like to ...
- 21
1
vote
1
answer
30
views
Testing the effect of a continious IV on DV, in order to explain group differences
I would like to ask a question regarding an analysis I’m planning and it might be a basic question so, apologies in advane.... To describe the situation: There are two groups of participants in my ...
- 31
3
votes
3
answers
195
views
How to perform a joint significance test for multiple coefficients across several regression models?
I have 10 regression models, each regressing a different dependent variable $ y_i $ (for $ i = 1, 2, \ldots, 10 $) on the same independent variable $ x $. Each model produces a coefficient for $ x $, ...
- 96
0
votes
0
answers
32
views
Appropriate Test for Measuring Change in Event Occurrence Over Time (RStudio)
I want to see if a training programme delivered to organisations to reduce the rate at which an event occurs actually has the intended effect of reducing said occurrence.
Each organisation that is ...
0
votes
0
answers
11
views
What test should I use for Weighted Mean Proportions?
I want to know if the "weighted mean proportion" of a racial group in the organization (overall) is significantly different from the "weighted mean proportion" of the group within ...
- 1
3
votes
1
answer
81
views
Evaluating the Validity of Election Results
In a suspicious election involving 5 candidates, the announced votes for each candidate are a multiple of 3. The chance of all five numbers being a multiple of 3 is only 0.41%. Therefore, some people ...
- 45
0
votes
0
answers
25
views
Unbiased test for homogeneity of means of exponenential samples
Given $K$ independent samples of $Y_{i1},\dots,Y_{in_i} \ \text{i.i.d.} \ \sim Exp(\lambda_i)$ with $i=1,\dots, K$ and $n_i$ the size of the $i$-th sample, is there any statistics with analytically ...
1
vote
0
answers
22
views
Testing the difference between two Root Mean Square Error values for statistical significance [duplicate]
I would like to compare the predictive power of 2 models. The models are meant to model count data and respective probabilities. I am using two metrics as means of comparison:
Root Mean Square Error
...
- 133
4
votes
1
answer
280
views
Breusch Pagan test in Python
The statsmodels module in Python provides a function to perform the Breusch Pagan test:
...
- 41
3
votes
1
answer
65
views
Testing dependence of two categorical variables with data separated by test subject
This relates to a real-world experiment I'm working on. Let's say in my experiment, test subjects look at pictures of sunglasses that are either red or blue, and then label them "cool" or &...
- 83
3
votes
1
answer
71
views
Testing for difference in trends of binary-parameter proportion/fraction versus continuous parameter between two (or more) different populations
I have three samples of objects: sample X (total = 138), sample Y (same basic class as X, but located in a different environment; total = 186), and sample Z (total = 996). For each object, I have a ...
- 203
0
votes
0
answers
8
views
Measuring statistical difference across two distributions of attention scores
For a study I'm conducting, I have two distributions of attention scores generated via a trained transformer model. I'm plotting the two distributions and I would like to run a statistical test to ...
- 659
3
votes
2
answers
313
views
sample size in chi-squared test
The chi-square test of independence is a type of non-parametric test, but in cases of small sample sizes, the Fisher's exact test should be used instead. My understanding of non-parametric methods is ...
- 103
2
votes
0
answers
29
views
What is the correct sample size for the purposes of this test?
$X_i$, $i = 1, \ldots, n_X$, are i.i.d. from some distribution; and $Y_j$, $j = 1, \ldots, n_Y$, are i.i.d. from some other distribution. Consider all pairs $(X_i, Y_j)$. The number of pairs is $n=n_X ...
- 1,251
0
votes
0
answers
16
views
Trying to find cointegrated pairs but getting false positives
Using statsmodels.tsa.stattools.coint to find cointegrated pairs of stock price time series but it seems to be returning a majority of false positives. Need ideas ...
- 1