# Questions tagged [p-value]

In frequentist hypothesis testing, the $p$-value is the probability of a result as extreme (or more) than the observed result, under the assumption that the null hypothesis is true.

2,846 questions
Filter by
Sorted by
Tagged with
38 views

### Can I use the multinomial test to estimate the probability that samples were derived from a distribution? [closed]

I need to perform some statistical reasoning in an automated tool. Because of this, the statistical processes need to be well-defined before seeing the exact data, and cannot be adapted manually if ...
1k views

### Can I use p-values without hypothesis?

I have been working on my Master's Thesis which included a survey and its results have been put through various tests. The unusual thing about my Thesis is that I have no hypotheses at all, making it ...
29 views

### How to statistically validate samplers? Is Chi-Square test relevant for synthetic data?

I am developing sampling procedures for different kinds of discrete objects (rankings, sets, etc.). In most cases I know the theoretical distribution of the objects I'm sampling. My idea is thus to ...
1 vote
62 views

### p-value comparison

Is it possible to compare two p-values? Suppose for a test with significance level 0.05, I get p = 0.06, and for another test with similar data, I get p = 0.20. What can I say if I want to make ...
4k views

### Why do we use hypothesis tests instead of just letting people do Bayesian updates?

Why do we need discretize our judgements using hypothesis tests? Why can't we just have people report the data every time a study is done, and the p-values and effect size, and then report how the ...
128 views

### Permutation test for exponential null hypothesis: really bad?

Having found nice formulas for testing the null hypothesis under exponentially-distributed samples, I wanted to see how well permutation tests could do the job. And the answer, assuming no mistakes, ...
20 views

### Choosing an appropriate statistical test in Multi-Reader Multi-Case (MRMC) Scenario

I am looking for the most suitable statistical test for the following situation: I am in a multi-reader multi-case (MRMC) scenario where 6 readers will perform binary classification on the same 165 ...
57 views

### Which type of t-test is best to use for my data

I'm looking for some guidance on which type of t-test is most suitable to use for my data. I want to compare the means of the variable 'Rate' between two groups in my data. I have 6 years of data and ...
864 views

### Are p-values computed from the a priori or a posteriori sampling distribution?

Wikipedia defines a p-value as follows: the p-value is the probability of obtaining test results at least as extreme as the result actually observed, under the assumption that the null hypothesis is ...
1 vote
53 views

### Asymptotics of $\mathbb E[-\log(p)]$ in a one-sample t-test as $n\to\infty.$

Consider a one-sample two-sided t-test, i.e. $X_1, \ldots, X_n$ are iid. $N(\mu, \sigma)$ random variables and we want to test $H_0\colon \mu=0$ versus $H_A\colon \mu\neq0$. The $t$-statistic is ...
6 views

### How to compare metrices in terms of RELATIVE bootstrap confidence interval

Situation: We are assessing several metrices to compare two distributions (e.g. ttest-pvalue, Wasserstein distance, binary classification metrices like ROC-AUC, etc.). One criteria should be the "...
47 views

### Do I always need to adjust the p value when using cor.test function in R? [closed]

I just came across a blog entry where a correlation matrix was manually calculated for some specific case. There, a p value adjustment was done afterwards (p.adjust() ) and I started to wonder if I ... 1 vote
99 views

### A good fit of the model but the estimates are very low

I'm trying to model soil quality using the soil quality index (SQI) as a function of earthworm abundance and elevation. The SQI is restricted to values between 0 and 1. I used multiple soil indicators ...
1 vote
49 views

### FDR correction - different number of p-values and models/hypotheses being tested

I have $3$ predictors: $1$ binary (prenatal alcohol exposure) $1$ categorical with $4$ categories (drinking patterns e.g., light stable, heavy reducer) $1$ continuous (total drinks consumed during ...
36 views

### High t statistic with high p value for same variable?

I am trying to do a difference in difference model to compare two cities' police departments' uses of force after treatment from a consent decree. When I run the model with clustered standard errors, ...
35 views

### Analysis of the difference between two groups on multiple outcomes

My question contains a few variations; I apologize for this in advance. I have two groups, and I have four outcome measures. These outcome measures are assessments of pain, function, depression level, ...
756 views

### Non-significant results when running Kruskal-Wallis, significant results when running Dwass-Steel-Critchlow-Flinger pairwise comparisons

I am writing my Master's thesis on adult's performance on cognitive and linguistic measures. I have four age groups and am investigating if there is an age difference in the performance on summarized ...
31 views

### Can you combine sequential testing with the Benjamini-Hochberg/Benjamini-Yekutieli procedure

I am trying to avoid large amounts of type-1 errors in the evaluation of multiple hypotheses within the same online A/B test. Currently, features are tested across different platforms, with several ...
92 views

### The use of p-values in tests of d-separation

In his very helpful online book on structural equation modeling, Jon Lefcheck writes the following concerning d-separation tests for SEMs: Once the model is fit, statistical independence is assessed ...
38 views

### is my p-value significant at 3.9e-11? [duplicate]

I know that if it says < 3.9e-11 it is significant because it is less than. But this time it has no < symbol, just the value of 3.9-11. Does this mean its not less than or is it still ...
68 views

### Bonferroni Post-hoc after normality test following a glm()

Experiment Design Mice belonging to knock-out ($n=7$) and wild-type ($n=6$) had their food intake recorded per hour across a 24 hour period. Within this 24 hour period, the first 12 hours is labelled ...
1 vote
46 views

### What do the p-values ​resulting from the SIMPER analysis in R represent?

I am doing a SIMPER analysis (in R) with the values ​​relating to individuals contacted in different reproductive seasons to see how the surveyed bird community has changed, and which species ...
38 views

### Multi-level modeling as a statistical criterion

I found Doordashe's article about conducting switchback experiments. In their article they use multi-level modeling instead of statistical criterions, which are usually used in A/B testing. They ...
1 vote
22 views

### Formula to calculate p-values for the fixed component of a mixed effects model

I’m building an algorithm to obtain a mixed effects model. So I am not using lme or lmer or anything. I was able to get my estimates and standard errors. Now I want to get the p-values for the fixed ...
44 views

### Hypothesis Testing - Varying p-value issue in my MWU

I have two machines (A & B) in our production line. A is older, B is very new. Both perform the same tasks, except that machine B is advanced and will perform 1 task less than the no of tasks in ...
490 views

### What is the information in an exact p-value?

Consider the following two statistical principles: 1) an exact test's $p$-value gives the exact frequency with which the observed random sample appears by chance, i.e., under a true null hypothesis; ...
414 views

### How many p-value observations do you think are required before doing FDR correction

I have two sets of correlation data and my variable of interest and I am wondering if/when o apply FDR to the obtained P-Values. On one hand, I have correlations with gene expression data, so a great ...
1 vote
100 views

### Regression model with good p-values but bad error

Suppose there is some data and all assumptions for regression are met. I fit 2 regression models now: Case 1: Regression model where all coefficients have significant $p$ values but overall the model ...
1 vote
46 views

### Why are the p-values for a binomial test very different from the p-values for a chi-squared test?

I am testing if a coin is fair by throwing it n-times and having n/2 + sqrt(n) heads. However, I get very different p-values ...
1 vote
96 views

### High p value - Low Effect size

What does it mean to have a high p value but a very very low effect size? What can I conclude? Is it necessary to report the effect size when p=0.998?
440 views

### Reducing threshold p-value in case of statistical tests following data exploration

Imagine, I got a new dataset. I look into it, trying to find a plausible hypothesis. During this I build charts, trying to find something interesting. Should I reduce the p-value threshold due to ...
29 views

### compare beta coefficient for two different regression models [duplicate]

0 I want to compare beta coefficient observed for a variable from two different models. Is there any method to do this? Let say beta coefficient of X1 is 0.6462 and beta coefficient for X1 in a ...
27 views

### Appropriate statistical test for stacked barplot?

I was wondering what would be an appropriate statistical test for the following stacked barplot. We are trying to compare how isoform expression (first image) and isoform proportion varies with ...
63 views

### Emmeans, pairs, contrasts, and all different p-values

The main regression formula I am using is as follows (Both the Group and Time variables are binary): ...
1 vote
59 views

### Can evidence of absence be obtained only in one-sided hypothesis tests?

It seems that if a one-sided Z-test returns a p-value $>1 - \alpha$, you have enough evidence of absence to reject the alternative hypothesis in favor of the null hypothesis, because if you were to ...
1 vote
41 views

### Do I need to use Bonferroni correction in commutation study?

I have a table like this: ...
51 views

### What does it mean when the p value is 0.000, Hazard ratio is 1 and 95% Confidence Interval is 1?

What does it mean when the p value is 0.000, Hazard ratio is 1 and 95% Confidence Interval is 1? From my understanding, a hazard ratio of 1 implies that there is equal hazard in both groups. A p-value ...
1 vote
296 views

### Is p-value correction necessary/required for this problem?

Suppose I have multiple regions of a tissue that consists of multiple subtypes of a tumor. I want to compare the significance of differences between these various tumor subtypes (i.e How they differ w....
23 views

### How to calculate correlation between two series with high autocorrelation

two series with high autocorrelation, so the degree of freedom is decline. Using t-test to estimate the p-value, they are easy to achieve a high significance level. How to adjusted the correlation ...
53 views

### how to calculate p-value based on # of trials and observed values?

In this article, the p-value is calculated as 0.32 for observing 55 heads in 100 coin flips. https://netflixtechblog.com/interpreting-a-b-test-results-false-positives-and-statistical-significance-...
88 views

### P-value and test power relationship

In this video the author says that Power of a test is the probability that we will correctly get a p-value below significance level My question is: when do we know that we correctly obtain a p-value ...
1 vote
66 views

### A question about the definition a p-values [duplicate]

In hypothesis testing, the definition of p value is the probability of obtaining test results at least as extreme as the results actually observed, under the assumption that the null hypothesis is ...
54 views

### How to calculate p-value when comparing the percentage similarity of matrices in R

My question is derived from a previous topic https://stackoverflow.com/questions/77287721/how-to-compare-several-matrices-and-calculate-the-percentage-difference-in-r/77287785#77287785 , where @...
59 views

### Proportion test under the null hypothesis

In the code below, two binomial responses are simulated with the same probability of success ($H_0$) and the p-value of the 1-sided z-test is stored. This is repeated 10000 times. ...
51 views

### How to find p-value from the Normal Distribution Table?

I'm reviewing a Topic about "Meaning of the p value and confidence interval" they gave an example about "The Coin Tossing Experiment" In the example I understand how to find "...
163 views

### When I change my reference level on my GLMER in R, why do the p-values change and why don't the estimates add up? Emmeans solution in answer

I am new to this. My study has three conditions (between subjects - low coordination, high coordination, high coordination with ostensive cues) and three repetitions of a game (within subjects - Game ...
436 views

### p value correction in multiple outcomes study

I am performing an analysis on different values of complement (in total 18, 9 in blood and 9 in CSF). We aim to analyze if any of these complement values: Differentiate between 3 diseases (Disease 1, ...
20 views

### How can I interpret this correlation? [duplicate]

I got correlation coefficient r = 0.33 but my p-value = 0.124 > 0.05 Is there a non-significant correlation or no correlation at all?
1 vote