Questions tagged [p-value]
In frequentist hypothesis testing, the $p$-value is the probability of a result as extreme (or more) than the observed result, under the assumption that the null hypothesis is true.
2,846
questions
0
votes
0
answers
38
views
Can I use the multinomial test to estimate the probability that samples were derived from a distribution? [closed]
I need to perform some statistical reasoning in an automated tool.
Because of this, the statistical processes need to be well-defined before seeing the exact data, and cannot be adapted manually if ...
- 1
5
votes
2
answers
1k
views
Can I use p-values without hypothesis?
I have been working on my Master's Thesis which included a survey and its results have been put through various tests. The unusual thing about my Thesis is that I have no hypotheses at all, making it ...
- 61
0
votes
0
answers
29
views
How to statistically validate samplers? Is Chi-Square test relevant for synthetic data?
I am developing sampling procedures for different kinds of discrete objects (rankings, sets, etc.). In most cases I know the theoretical distribution of the objects I'm sampling. My idea is thus to ...
- 1
1
vote
1
answer
62
views
p-value comparison
Is it possible to compare two p-values? Suppose for a test with significance level 0.05, I get p = 0.06, and for another test with similar data, I get p = 0.20. What can I say if I want to make ...
- 83
34
votes
9
answers
4k
views
Why do we use hypothesis tests instead of just letting people do Bayesian updates?
Why do we need discretize our judgements using hypothesis tests?
Why can't we just have people report the data every time a study is done, and the p-values and effect size, and then report how the ...
- 451
6
votes
1
answer
128
views
Permutation test for exponential null hypothesis: really bad?
Having found nice formulas for testing the null hypothesis under exponentially-distributed samples, I wanted to see how well permutation tests could do the job. And the answer, assuming no mistakes, ...
- 1,060
0
votes
0
answers
20
views
Choosing an appropriate statistical test in Multi-Reader Multi-Case (MRMC) Scenario
I am looking for the most suitable statistical test for the following situation:
I am in a multi-reader multi-case (MRMC) scenario where 6 readers will perform binary classification on the same 165 ...
- 1
2
votes
2
answers
57
views
Which type of t-test is best to use for my data
I'm looking for some guidance on which type of t-test is most suitable to use for my data.
I want to compare the means of the variable 'Rate' between two groups in my data. I have 6 years of data and ...
- 21
6
votes
5
answers
864
views
Are p-values computed from the a priori or a posteriori sampling distribution?
Wikipedia defines a p-value as follows:
the p-value is the probability of obtaining test results at least as extreme as the result actually observed, under the assumption that the null hypothesis is ...
- 223
1
vote
0
answers
53
views
Asymptotics of $\mathbb E[-\log(p)]$ in a one-sample t-test as $n\to\infty.$
Consider a one-sample two-sided t-test, i.e. $X_1, \ldots, X_n$ are iid. $N(\mu, \sigma)$ random variables and we want to test $H_0\colon \mu=0$ versus $H_A\colon \mu\neq0$.
The $t$-statistic is ...
- 151
0
votes
0
answers
6
views
How to compare metrices in terms of RELATIVE bootstrap confidence interval
Situation: We are assessing several metrices to compare two distributions (e.g. ttest-pvalue, Wasserstein distance, binary classification metrices like ROC-AUC, etc.). One criteria should be the "...
- 450
0
votes
1
answer
47
views
Do I always need to adjust the p value when using cor.test function in R? [closed]
I just came across a blog entry where a correlation matrix was manually calculated for some specific case. There, a p value adjustment was done afterwards (p.adjust() ) and I started to wonder if I ...
JustMe
1
vote
1
answer
99
views
A good fit of the model but the estimates are very low
I'm trying to model soil quality using the soil quality index (SQI) as a function of earthworm abundance and elevation. The SQI is restricted to values between 0 and 1. I used multiple soil indicators ...
1
vote
1
answer
49
views
FDR correction - different number of p-values and models/hypotheses being tested
I have $3$ predictors:
$1$ binary (prenatal alcohol exposure)
$1$ categorical with $4$ categories (drinking patterns e.g., light stable, heavy reducer)
$1$ continuous (total drinks consumed during ...
- 11
0
votes
1
answer
36
views
High t statistic with high p value for same variable?
I am trying to do a difference in difference model to compare two cities' police departments' uses of force after treatment from a consent decree. When I run the model with clustered standard errors, ...
0
votes
0
answers
35
views
Analysis of the difference between two groups on multiple outcomes
My question contains a few variations; I apologize for this in advance.
I have two groups, and I have four outcome measures. These outcome measures are assessments of pain, function, depression level, ...
- 31
8
votes
3
answers
756
views
Non-significant results when running Kruskal-Wallis, significant results when running Dwass-Steel-Critchlow-Flinger pairwise comparisons
I am writing my Master's thesis on adult's performance on cognitive and linguistic measures. I have four age groups and am investigating if there is an age difference in the performance on summarized ...
- 91
0
votes
0
answers
31
views
Can you combine sequential testing with the Benjamini-Hochberg/Benjamini-Yekutieli procedure
I am trying to avoid large amounts of type-1 errors in the evaluation of multiple hypotheses within the same online A/B test. Currently, features are tested across different platforms, with several ...
3
votes
1
answer
92
views
The use of p-values in tests of d-separation
In his very helpful online book on structural equation modeling, Jon Lefcheck writes the following concerning d-separation tests for SEMs:
Once the model is fit, statistical independence is assessed ...
- 339
0
votes
0
answers
38
views
is my p-value significant at 3.9e-11? [duplicate]
I know that if it says < 3.9e-11 it is significant because it is less than. But this time it has no < symbol, just the value of 3.9-11. Does this mean its not less than or is it still ...
- 1
2
votes
1
answer
68
views
Bonferroni Post-hoc after normality test following a glm()
Experiment Design
Mice belonging to knock-out ($n=7$) and wild-type ($n=6$) had their food intake recorded per hour across a 24 hour period. Within this 24 hour period, the first 12 hours is labelled ...
- 45
1
vote
1
answer
46
views
What do the p-values resulting from the SIMPER analysis in R represent?
I am doing a SIMPER analysis (in R) with the values relating to individuals contacted in different reproductive seasons to see how the surveyed bird community has changed, and which species ...
2
votes
1
answer
38
views
Multi-level modeling as a statistical criterion
I found Doordashe's article about conducting switchback experiments. In their article they use multi-level modeling instead of statistical criterions, which are usually used in A/B testing. They ...
- 23
1
vote
0
answers
22
views
Formula to calculate p-values for the fixed component of a mixed effects model
I’m building an algorithm to obtain a mixed effects model. So I am not using lme or lmer or anything.
I was able to get my estimates and standard errors.
Now I want to get the p-values for the fixed ...
- 47
0
votes
0
answers
44
views
Hypothesis Testing - Varying p-value issue in my MWU
I have two machines (A & B) in our production line. A is older, B is very new. Both perform the same tasks, except that machine B is advanced and will perform 1 task less than the no of tasks in ...
- 3
6
votes
4
answers
490
views
What is the information in an exact p-value?
Consider the following two statistical principles: 1) an exact test's $p$-value gives the exact frequency with which the observed random sample appears by chance, i.e., under a true null hypothesis; ...
- 508
4
votes
3
answers
414
views
How many p-value observations do you think are required before doing FDR correction
I have two sets of correlation data and my variable of interest and I am wondering if/when o apply FDR to the obtained P-Values.
On one hand, I have correlations with gene expression data, so a great ...
- 111
1
vote
6
answers
100
views
Regression model with good p-values but bad error
Suppose there is some data and all assumptions for regression are met. I fit 2 regression models now:
Case 1: Regression model where all coefficients have significant $p$ values but overall the model ...
1
vote
1
answer
46
views
Why are the p-values for a binomial test very different from the p-values for a chi-squared test?
I am testing if a coin is fair by throwing it n-times and having n/2 + sqrt(n) heads.
However, I get very different p-values ...
- 13
1
vote
1
answer
96
views
High p value - Low Effect size
What does it mean to have a high p value but a very very low effect size? What can I conclude? Is it necessary to report the effect size when p=0.998?
- 11
4
votes
3
answers
440
views
Reducing threshold p-value in case of statistical tests following data exploration
Imagine, I got a new dataset. I look into it, trying to find a plausible hypothesis. During this I build charts, trying to find something interesting. Should I reduce the p-value threshold due to ...
- 1,575
0
votes
0
answers
29
views
compare beta coefficient for two different regression models [duplicate]
0
I want to compare beta coefficient observed for a variable from two different models. Is there any method to do this? Let say beta coefficient of X1 is 0.6462 and beta coefficient for X1 in a ...
0
votes
0
answers
27
views
Appropriate statistical test for stacked barplot?
I was wondering what would be an appropriate statistical test for the following stacked barplot. We are trying to compare how isoform expression (first image) and isoform proportion varies with ...
- 11
0
votes
1
answer
63
views
Emmeans, pairs, contrasts, and all different p-values
The main regression formula I am using is as follows (Both the Group and Time variables are binary):
...
- 119
1
vote
0
answers
59
views
Can evidence of absence be obtained only in one-sided hypothesis tests?
It seems that if a one-sided Z-test returns a p-value $>1 - \alpha$, you have enough evidence of absence to reject the alternative hypothesis in favor of the null hypothesis, because if you were to ...
- 111
1
vote
2
answers
41
views
0
votes
1
answer
51
views
What does it mean when the p value is 0.000, Hazard ratio is 1 and 95% Confidence Interval is 1?
What does it mean when the p value is 0.000, Hazard ratio is 1 and 95% Confidence Interval is 1? From my understanding, a hazard ratio of 1 implies that there is equal hazard in both groups. A p-value ...
- 1
1
vote
2
answers
296
views
Is p-value correction necessary/required for this problem?
Suppose I have multiple regions of a tissue that consists of multiple subtypes of a tumor. I want to compare the significance of differences between these various tumor subtypes (i.e How they differ w....
- 11
0
votes
0
answers
23
views
How to calculate correlation between two series with high autocorrelation
two series with high autocorrelation, so the degree of freedom is decline. Using t-test to estimate the p-value, they are easy to achieve a high significance level. How to adjusted the correlation ...
- 1
0
votes
1
answer
53
views
how to calculate p-value based on # of trials and observed values?
In this article, the p-value is calculated as 0.32 for observing 55 heads in 100 coin flips.
https://netflixtechblog.com/interpreting-a-b-test-results-false-positives-and-statistical-significance-...
- 530
2
votes
1
answer
88
views
P-value and test power relationship
In this video the author says that
Power of a test is the probability that we will correctly get a p-value below significance level
My question is: when do we know that we correctly obtain a p-value ...
- 21
1
vote
1
answer
66
views
A question about the definition a p-values [duplicate]
In hypothesis testing, the definition of p value is the probability of obtaining test results at least as extreme as the results actually observed, under the assumption that the null hypothesis is ...
- 11
0
votes
1
answer
54
views
How to calculate p-value when comparing the percentage similarity of matrices in R
My question is derived from a previous topic
https://stackoverflow.com/questions/77287721/how-to-compare-several-matrices-and-calculate-the-percentage-difference-in-r/77287785#77287785
, where @...
- 211
3
votes
1
answer
59
views
Proportion test under the null hypothesis
In the code below, two binomial responses are simulated with the same probability of success ($H_0$) and the p-value of the 1-sided z-test is stored. This is repeated 10000 times.
...
- 2,173
0
votes
0
answers
51
views
How to find p-value from the Normal Distribution Table?
I'm reviewing a Topic about "Meaning of the p value and confidence interval" they gave an example about "The Coin Tossing Experiment"
In the example I understand how to find "...
2
votes
1
answer
163
views
When I change my reference level on my GLMER in R, why do the p-values change and why don't the estimates add up? Emmeans solution in answer
I am new to this. My study has three conditions (between subjects - low coordination, high coordination, high coordination with ostensive cues) and three repetitions of a game (within subjects - Game ...
5
votes
3
answers
436
views
p value correction in multiple outcomes study
I am performing an analysis on different values of complement (in total 18, 9 in blood and 9 in CSF). We aim to analyze if any of these complement values:
Differentiate between 3 diseases (Disease 1, ...
0
votes
1
answer
20
views
How can I interpret this correlation? [duplicate]
I got correlation coefficient r = 0.33
but my p-value = 0.124 > 0.05
Is there a non-significant correlation or no correlation at all?
1
vote
1
answer
65
views
If two distributions are sufficiently close to each other, how is it possible for the difference between them to be statistically significant?
As I understand it, p-values are derived from a test statistic, which measures how unlikely it is that we would get those data, assuming the null hypothesis is true.
But if this is the case, would it ...
- 23
0
votes
0
answers
19
views
How to describe data? With p-value or without it?
I have health related data sets such as age specific mortality of all deaths in specified areas and time periods. Since I have anonymised records for each event, I can calculate indicators of central ...