# Questions tagged [p-value]

In frequentist hypothesis testing, the $p$-value is the probability of a result as extreme (or more) than the observed result, under the assumption that the null hypothesis is true.

2,929 questions
Filter by
Sorted by
Tagged with
383 views

### Related low p-values that do not meet statistically significant thresholds

If I were to have multiple related measures, each with "low" p-values that do not quite meet the threshold of significance, could the fact that multiple measures produce a general trend with ...
14 views

### Significance of the coefficients of a multinomial mixed regression built with npmlt function

I fitted (using npmlt function) a mixte multinomial regression model which ran without errors. I expected that the outputs of the model include p-values that I didn't find. My need now is that someone ...
38 views

### Hypothesis testing on sum and product of beta random variables

In population genomics, a locus at which all the individuals of a population have alleles equal to the reference allele is called a fixed locus. I am working on a model to obtain the expected number ...
• 93
16 views

### My questions are related to the outputs of model built with npmlt function [closed]

I fitted (using npmlt function) a mixte multinomial regression model which ran without errors. I expecting that the outputs of the model includes p-values that I didn't find. My need now is that ...
41 views

### Test whether my new method's type I error rate correctly matches chosen alpha

I am testing two different methods for hypothesis testing, bootstrapped likelihood ratio tests, and likelihood ratio tests assuming an asymptotic distribution. I would like to see if the number of ...
143 views

### Are there alternative statistics to a p-value in NHST?

Sometimes, when the logic of p-value is explained and motivated, it happens in the following way: The concept of a random sample is introduced, together with the need to consider the possibility that ...
• 753
26 views

### Determining the p-value of a test statistic, which is not distributed according to a commonly known distribution under the null hypothesis

Currently I am working in R on a project that aims to identify Dragon King events (massive outliers) in large datasets. These outliers appear for example in the city sizes in England, where London is ...
48 views

### Testing for difference in trends of binary-parameter proportion/fraction versus continuous parameter between two (or more) different populations

I have three samples of objects: sample X (total = 138), sample Y (same basic class as X, but located in a different environment; total = 186), and sample Z (total = 996). For each object, I have a ...
• 153
22 views

### The problem of double adjustment of p-values in multilevel models with categorical predictors

I have a set of around 30 multilevel models which have the same predictor variables (and the same cluster variable), but different outcome variables. Since there are so many models, I want to perform ...
• 357
18 views

I want to determine the correlation and p-value between one variable and all of the others in an R data frame, so I created a correlation matrix with cor_mat, and ...
1 vote
35 views

### Why can we use RMSE as a proxy for the effect size of a VAR or Granger Causality?

My question here follows a previous question by me here: Is there a meaningful option to compute something like an effect size for Granger Causality?. In this previous question I asked for options to ...
• 307
49 views

### Is there a meaningful option to compute something like an effect size for Granger Causality?

I am running Granger Causality (GC) analysis using statsmodels.tsa.stattools.grangercausalitytests between several time-series pairs. The time-series pairs used for ...
• 307
1 vote
41 views

### Comparison of two test metrics

I'm trying to compare two test metrics (Metric A and Metric B) to determine which one better predicts a delta value, which represents a Euclidean difference. I am unsure how to determining which ...
• 41
492 views

### Variable is significant in multiple linear regression but not in t-test of the subgroups

I am getting different p-values for a variable in t-test and its coefficient in multiple linear regression so I am unsure which one to believe. As an example, my hypothetical dataset consists of ...
392 views

### Ideal test/p-value calculation for difference in means with small sample size and right skewed data? [closed]

I need to compare a response variable between two different groups. My control group has 205 observations while my comparison group only has 2 observations (unfortunately this is the nature of the ...
30 views

### How to measure Statistical Significance for calculated Sensitivity, Specificity, Precision, Accuracy and f1 values?

I am trying to understand how to calculate one or more measures of statistical significance to display alongside metrics I've calculated from my data. Abbreviations I am using in the rest of this post:...
218 views

### Estimating effects in the presence of a mediator

Suppose that one is interested to compare the effect of biological age versus "cognitive age" on a variety of outcomes. Cognitive age is measured by testing intelligence. The outcomes are a ...
• 753
20 views

### Why Do Different MANOVA Test Statistics Produce the Same Approximate F and P Values in R?

I am using the manova() function in R to perform multivariate analysis of variance on my dataset. I have tried using different test statistics such as Pillai's ...
114 views

### What is the proper formatting of very low p values? [closed]

Generally, I believe in giving exact p values rather than something like " p < 0.05". But, sometimes, the p value is extremely low, on the order, say, of $10^{-10}$ or even lower. Under ...
• 125k
31 views

### Sufficient number of data to determine whether a model fits the data well

I was wondering if, chosen a proper mathematical model, there is a minimum number of data that allows us to state if the model fits well the data or it doesn't. I'll explain better my question. Let's ...
1k views

### Why do physicists use sigma while biologists use p values/posterior probabilities?

I work in the analysis of biological data, and am familiar with the use of maths to determine a test statistic, and the comparison of this test statistic with a threshold to evaluate a hypothesis. ...
• 231
51 views

### Help interpret these P-values [duplicate]

What does it mean when the model P-value comes out larger than alpha but the P-value for one of the main effects comes out smaller than alpha? The actual ANOVA is below. Alpha is 0,0027.
45 views

### Are the p-values obtained on the same sample using synthetic AA tests (Monte Carlo) independent values?

Let's say we have the following procedure. We take a fixed sample of size n and perform the procedure 1000 time: we divide (split) it equally into 2 groups; we calculate p value using the F function (...
35 views

### Different P-Value and AIC before/after standardization [Python - Statsmodels]

I am investigating the correlation between environmental variables (15 continuous variables grouped as 'DHIs' in the code below) and fox occurrence (binary), using logistic regression / Python ...
54 views

### Am I using the fisher.test() function incorrectly? Can't replicate p-value found in research paper

I've been trying to replicate the results of this research paper (The Tale of Cochran’s Rule: My Contingency Table has so Many Expected Values Smaller than 5, What Am I to Do? by Kroonenberg and ...
1 vote
41 views

### How do I do the p-adjustement in statistical tests?

context: a study about which type of bread and/or filling attracts ants the most I have a table with 3 columns : the number of ants attracted, type of bread, type of filling. The question is : do ...
13 views

### Which statistical test to use for comparison of influence of two independent variables?

I am conducting a research of "Comparison of Influence of Parents' and Teacher's Attitude Towards Math to Anxiety of Students in Math" Now my null hypothesis: There is no difference in ...
35 views

### How can I combine p-values estimated for individual variables within a single dataset using different statistical methods?

I'm analyzing a multivariate dataset with two classes (case/control). I've calculated p-values for each variable to test their discrimination perfomance using two different methods: 1) univariate t-...
2k views

### Having a second bite of the data-apple without p-hacking

My hypothesis concerns intervention versus control in a randomised controlled trial (between-subjects, n=500 per group, online survey experiment). I pre-registered that my primary test would be a ...
• 913
1 vote
39 views

### Can the One-Sided Mann-Whitney U Test Have a Composite Null Hypothesis?

Let's say there's an i.i.d. sample $X_1, X_2, \ldots, X_{n_1}$ from a distribution with the CDF $F$ and $Y_1, Y_2, \ldots, Y_n$ from a distribution whose CDF is $G$. As far as I know, the null ...
• 1,099
86 views

### Interpretation of high p value of a coefficient in linear regression [duplicate]

In a medical article, the multiple linear regression model included some coefficients with low p values but also some with p>0.05. How can I interpret these high p values and should they be be ...
67 views

### Coefficient in Regression Analysis

I am running regression on two variables (independent and dependent). If the results show p-value > 0.05, it is not statistically significant. Does this mean we reject the coefficient as well? If ...
76 views

### untrustworthy p values? emtrends following lmms with fixed effects of different levels of repetition

Edited to clarify further 23/04/24: I have a fairly complex linear mixed effect model in R and I believe that there are issues to do with repeated data when running my post hoc tests. The model ...
31 views

### Difference between a comparison of independent variables t.test and linear regression model of one category?

The thing is I am trying to obtain confidence intervals of several variables (food, blood chemical analysis...) of patients who are divided in two groups according the intervention. Until now I was ...
1 vote
21 views

### Benjamini Hochberg Procedure [closed]

I am working on a problem for class related to multiple testing where I would like to run the BH procedure with a known $\pi_{0}$, denoting the proportion of hypothesis that are truly null, given ...
• 345
264 views

### Why was the term "significance" ($\alpha$) chosen for the probability of Type I error?

I'm currently studying "Statistics 1" as part of my Computer Science degree, and I'm having trouble understanding the concept of "significance." We were provided with the following ...
• 93
17 views

### How to test for statistical significance on a multi-country dataset?

I am attempting to test for statistical significance between treatment and control groups, with data from 7 countries, using a Welch 2-sample t-test [t.test() in R]. 6 out of 7 countries have ...
852 views

### Regressors Became Statistically Insignificant Upon Correcting for Autocorrelation

I am using Stata and used the regress command and received $p$ values that indicate the regressor is statistically significant. However, after plotting the residuals, I noticed there was clearly an ...
68 views

### Understanding of nominal p-values in sample size calculation

I need your help to understand the R-Output. In this example, two interim analyses were planned. The adjusted p-values are shown below. However, I am still not clear what the nominal p-values mean. ...
82 views

### Why p-values as $P(p≤\alpha | H=1)≤\alpha$ and not $P(H=1 | p≤\alpha)$?

That's my question, as if I start from the definition of a $p$-value ($P(p≤\alpha | H=1)≤\alpha$ ) and apply Bayes rule I arrive to:  P(p≤\alpha | H=1)=P(H=1 | p≤\alpha)\frac{P(p≤\alpha)}{P(H=1)} ≤\...
• 123
113 views

### Statistical Thresholds for RNA-Seq Analysis: Considering the Intersection of Treatment Effects

I'm conducting an RNA-seq experiment involving worms exposed to chemicals A, B, and A+B to observe mRNA changes due to exposure. I'm particularly interested in identifying mRNA changes in the A+B ...
91 views

### Fisher's exact test - how to find tables "as extreme"

I attempted to understand the computation of the Fisher's exact test through programming using this formulation: The Fisher exact test computes the p-values by finding the probabilities of all ...
1 vote
39 views

### What is a good p-value adjustment for a permutational analysis like pairwise PERMANOVA?

I am comparing plant community responses to experimental treatments using the pairwise.permanova function in R. However, there are no clear answers regarding which ...
• 11
1 vote
18 views

### Imputing a Continuous P-value Distribution from Discrete P-values [closed]

I'm exploring methods to create a continuous-like P-value distribution from discrete P-values. For example, consider the process where a balanced coin is tossed four times per experiment to note the ...
• 143
1 vote
47 views

### Comparing sequence of unfair coin flips with different predicted probabilities

Suppose we have $n$ independent coins, each of which has an unknown and different probability of coming up heads. We have a magical machine that guesses the probabilities of each coin coming up heads. ...
29 views

### Comparing averages of two groups

Say I want to check if the averages of 2 groups (A and B) are the same. For each member in group A, there are some observations; same applies for group B. If I want to take the average of A and B, I ...
228 views

### Can anyone work out the P value from this graph?

Can anyone work out the p value just based on the graph below? I cannot understand how this is calculated and what additional information I would needs to work it out.
62 views

### Relationship between p-value and Cramer's V

Is there a relationship between p-value and Cramer's V value after conducting Chi-square test for independence... is for Cramer's V value of about 0.13 the associated p-value is of about 0.001 or it ...
1 vote