Questions tagged [p-value]
In frequentist hypothesis testing, the $p$-value is the probability of a result as extreme (or more) than the observed result, under the assumption that the null hypothesis is true.
2,971 questions
4
votes
4
answers
552
views
How to permute p-values?
Recently, my colleague encountered a problem while working with a dataset, df_a, which contains gene data. Each row represents a ...
- 153
0
votes
0
answers
12
views
Literature on the usage of multiple regressions to validate an hypothesis using multiple variations of variables
I'm trying to show that in an equation of the form var1 ~ var2 + var3 + ... + varX, var2 has a has a statistically significant effect. However, there are many ways of defining var1, var3... varX (var ...
2
votes
3
answers
208
views
Interpret p-value in A/B test results
I need to interpret the A/B test results. So, I have the control cohort named Baseline and also I have two other cohorts named ...
- 119
0
votes
0
answers
33
views
Multiple Testing p-value Adjustment and Preservation of Significant Differences
When adjusting for multiple hypothesis tests, a non-uniform distribution of p-values that has a promising-looking Q-Q plot ends up with all adjusted p-values far above 0.05. Are there other multiple ...
- 211
5
votes
1
answer
112
views
Likehood ratio test vs wald test multicolinearity
I want to do a project about the display how robust likehood ratio testing can be. My idea is to show that in a logistic regression where one predictor displays multicolinearity the wald test show ...
- 51
0
votes
0
answers
118
views
Which significance test to use for a difference of means? How to be sure?
[Updated] I have two data sets containing all official recorded games played in the USA for this one sport. One data set is for the women's teams and the other for the men's teams. I'm trying to ...
0
votes
0
answers
13
views
p values one vs two tailed Kendall's tau, Spearman's rho, Pearson r
For Pearson correlations one can easily determine the p values for one tailed tests from the p values of two tailed tests by
p1tailed, predicted direction = 1/2 * p2tailed
p1tailed, opposite ...
- 1,407
0
votes
1
answer
54
views
Getting NAN p-value from logistic regression!
I'm trying to find p-values for coefficients of a logistic regression using MATLAB fitglm function, but I keep getting NAN values. I suspect this caused by the INF ...
- 121
0
votes
0
answers
27
views
When using entire population statistics, are p-values, confidence intervals, and standard errors still relevant? [duplicate]
I’m analyzing the full set of arbitration cases under UNCITRAL (the entire population, not just a sample) using OLS regression. Since I have data on every case, I’m wondering whether traditional ...
2
votes
2
answers
36
views
Paired sample (data) with unequal sample size
I am reading a publication in which the author mentioned that they used the paired t-test. In their study, a questionnaire was provided to a group of people before and after surgery to check the ...
- 155
3
votes
0
answers
96
views
Isn't it a problem to conduct post-hoc tests based on the overall p-value from ANOVA?
Let's say that after conducting an ANOVA, I get an overall p-value smaller than 0.05 (or whatever alpha level). Following that, I decide to conduct post-hoc pairwise tests to check the individual ...
- 31
14
votes
8
answers
2k
views
Do we reject null if we have a pvalue of 0.0503 at alpha = 0.05? What way to validate the rejection, if ever?
I know it is more than 0.05, but I'm just wondering since rounding it to two decimals will give 0.05. I just wanted to ensure I am not falsely accepting the null hypothesis. Is there any way to say ...
- 143
2
votes
0
answers
65
views
Doing "maximum p-value estimation" instead of maximum likelihood
Whenever we do maximum likelihood estimation, we look for the parameters that maximize the probability density of the data. On the other hand, when we compute p-values, we look at the tail probability ...
- 475
9
votes
5
answers
729
views
p-values for hypothesis testing
Intuitively, the p-value in a standard hypothesis test (where the test statistic is either normally, or approximately normally distributed) is thought of as the "probability of observing a value ...
- 949
4
votes
1
answer
226
views
Reviewer’s questions about the baseline analysis
Research Context:
I conducted a clinical study to explore the correlation between variable A and the incidence of heart disease, using a sample size of 4,000 individuals. The sample was divided into ...
- 147
5
votes
2
answers
271
views
Is hierachical regression with aggressive p-deletion really much 'better' than stepwise?
In many medical science fields "hierarchical regression" is a popular method.
The approach is to break variables into categories, add one category of variables at a time and then remove ...
- 203
1
vote
1
answer
37
views
Is a binomial of Chi-Square or t test the correct test to compare dichotomous variables?
You have an ML model that outputs 100,000 out of a million observations of a dichotomous variable 0 or 1. You want to see whether the distribution of the outputted 100,000 observations of the model is ...
- 21
0
votes
1
answer
34
views
How to interpret p-value?
I have a dataset of students, with mock-final and final tests.
With hypotheses,
$H_0: \mu_{mock-final} = \mu_{final}$
$H_0: \mu_{mock-final} \ \neq \mu_{final}$
I started with 2-tailed t-test ...
- 103
11
votes
4
answers
647
views
Generally, are we supposed to remove all nonsignificant paths in an SEM analysis?
Given that we favour parsimony in science, should we remove nonsignificant paths that are nonsignificant if they do not result in significantly worse model fit?
After we remove the nonsignificant ...
- 331
8
votes
5
answers
938
views
Can one use the p-value to perform hypothesis testing instead of comparing the test statistic to the critical value at a given significance level?
In hypothesis testing, if a question states the level of significance, which is alpha, then does it necessarily mean that we have to use the classical method? (Means finding critical value and then ...
3
votes
3
answers
110
views
How do I calculate the posterior probability of a hypothesis given the result of the test?
I understand that the p-value is the probability of obtaining the data (or more extreme values) if the null hypothesis is true. What researchers really want to know, however, is the probability that ...
- 321
2
votes
1
answer
43
views
Is $T_{initial} = T_{permuted}$ tallied as significant or insignificant in permutation testing?
Assume that I perform a permutation test to check that two distributions are the same, and the output test statistics are always integers. There is no single direction so the test will be two tailed. ...
- 695
16
votes
2
answers
1k
views
Does every variable need to be statistically significant in a regression model?
I recently fit a regression model (ARIMAX) in which some variables (3) were statistically significant and some were not (1). I removed the statistically insignificant variables and refit the model, ...
- 315
7
votes
3
answers
220
views
Finding p-value with decimal degree of freedom
I am learning Welch's t-test and I successfully found that, in a two sample test, $t=2.968$ and the degree of freedom is $5.1$. I then go to an online p-value calculator for the p-value at 0.05 ...
- 73
2
votes
1
answer
59
views
Discrepancy Between KS Test Results and CDF Visualizations in Neuronal Correlation Analysis
Discrepancy Between KS Test Results and CDF Visualizations in Neuronal Correlation Analysis
I am analyzing Pearson Correlation Coefficient (PCC) values computed pairwise for a set of neurons measured ...
- 43
11
votes
5
answers
2k
views
What did statisticians use to quickly find p-values before applets were popularized in the late 2010s? [closed]
New statistics student here, I found out from my professor today that actually, applets that simulate repetitions of samples weren't really around until the late 2010s, which was really surprising to ...
4
votes
1
answer
105
views
How to adjust for p-values in Kruskal Wallis test with two or more hypothesis
I need help adjusting for p-values. I have a cohort of 200 patients. I am using 3 metrics to divide this group into different categories. Here is how they are distributed.
Metric A Category
Number of ...
- 41
3
votes
2
answers
77
views
When to Apply P-Value Corrections for Multiple Testing: A Multi-Outcome Study Example
Suppose I want to examine the association between a total score and independent binary variables (e.g., sex) and multi-categorical variables (e.g., grade). I conducted a t-test for the binary ...
- 71
10
votes
5
answers
940
views
Confused about the uniform distribution of p-values under the null hypothesis
I'm currently taking an introductory bioinformatics course, and I'm confused about the uniform distribution of p-values under the null hypothesis (when the null hypothesis is true).
It's been shown ...
- 195
0
votes
0
answers
19
views
Bootstrap confidence and p-values
Conducting bootstrap to obtain confidence interval is pretty standard (e.g., coefficient in regression and adjust for adjusting for selection).
Since we obtain a confidence interval, we can also use ...
- 1,465
3
votes
2
answers
302
views
Is downsampling a valid approach to compare regression results across groups with different sample sizes? If so, how?
I have a large dataset of countries, each categorized into one of two regions, and I'm interested in how the effect of a predictor variable
𝑋 differs between the two regions. I plan to fit separate ...
1
vote
1
answer
36
views
p-values based on clustered se with glm
I would like to retrieve the p-values from glm with clustered standard errors. I have used a function to calculate the clustered standard errors and then use these to generate a summary of the model ...
- 11
0
votes
1
answer
35
views
How do I calculate the p-value for two success rates where one of the test sets is a subset of the other?
I know questions like this have been asked a hundred times but unfortunately I'm not smart enough to figure out which answer applies to my problem.
For context: I have an image classification model ...
- 9
2
votes
1
answer
45
views
Should All Variables Have Adjusted P-Values in Multiple Hypothesis Testing or Just Hypotheses-Relevant Variables?
I'm currently working on a project involving multiple hypothesis testing. I understand the importance of adjusting p-values to control for Type I errors, but I'm unsure whether I should adjust the p-...
- 21
-1
votes
1
answer
38
views
Paramaterization of the Normal Distribution in Hypothesis Testing
When looking at how a z-statistic fits into the normal distribution $f(x) = \frac{1}{\sigma \sqrt{2\pi} } e^{-\frac{1}{2}\left(\frac{x-\mu}{\sigma}\right)^2}$ my intuition was always that it was just ...
- 3
0
votes
1
answer
39
views
P-value of 0 when comparing two treatments using bootstrap method
I am comparing two treatments and doing a t-test or a Wilcoxon test I get p-value of 2.2e-16.
I wanted to try bootstrapping because the samples are large (over 1000 ...
0
votes
0
answers
14
views
Should I Apply FWER Correction After Performing the Dwass-Steel-Crichtlow-Fligner Post-Hoc Test?
I have a scenario where I need to compare multiple groups ($N > 2$) with non-Gaussian distributions. After performing the Kruskal-Wallis test and finding a significant p-value ($p < \alpha$), I ...
1
vote
2
answers
179
views
Choosing p-adjustment method in pairwise Wilcox test
I need some help in choosing p-adjustment method in my pairwise wilcox test on weights in four different independent groups (using R).
I have been reading about the different methods and googled ...
1
vote
1
answer
63
views
Determining statistical significance
I am trying to determine the statistical significance of my results at a 95 percent confidence level. I have the coefficients and the standard errors. I wanted to check if the following statement is ...
-2
votes
1
answer
78
views
Chi-square test on contingency tables, with same input for both arguments of the test [closed]
If I use the same input for both arguments of the chisq.test (in R), shouldn't I get a p-value equal to 1? I get instead 0.
- 444
2
votes
2
answers
75
views
In log-rank testing how could the CI cross 1 with a p-value < .05?
Came across these results comparing standard therapy with standard therapy and atezolizumab in endometrial cancer.
Findings state survival in placebo group vs treatment as HR 0·82, 95% CI 0·63–1·07; ...
- 75
0
votes
0
answers
40
views
Combining p-values from bootstrap iterations
I am trying to model the probability of a mutation with respect to a continuous response variable $x$. I am building a logistic regression model with mutation status of each gene as $y$, and $x$ as a ...
- 13
4
votes
2
answers
545
views
Odds ratio and p-value
Is it possible that in logistic regression model, one variable has high odds ratio and higly significant and other variable which has low odds ratio but is also higly significant.
- 41
0
votes
0
answers
36
views
Is it better to use the W or p-value to determine normality using Shapiro-Wilk?
I have read somewhere that W values above 0.9 are considered normal. So I wanted to use that as a cutoff for normality. However, upon running various scenarios I have come up with the result of W = 0....
1
vote
0
answers
38
views
p values for a ARMA - GARCH model with exogenous variable
I am trying estimate the parameters for the following ARMA(1, 2) - GARCH(1, 1) model, with an exogenous variable as well. The model specification is as follows:
$ x_t = \mu + \beta_Y \cdot y_{t-1} + ...
- 21
1
vote
1
answer
73
views
Can I use the Estimate of a model directly without considering the p-value?
This is a follow-up question of (About Multivariate Generalized Regression Mixed Models with negative binomial distribution). The research question is to get the intraindividual (within-subject) ...
- 301
3
votes
2
answers
249
views
Confused about the p-value and Cohen's d
I have the following code that computes both the p-value and cohen's distance.
...
- 61
1
vote
2
answers
64
views
Misunderstanding p-value [duplicate]
I read that the p-value is the likelihood that a number is equal to or greater than the actual observed number if the null hypothesis is correct 1 . I understand the null hypothesis means that there’s ...
- 111
3
votes
2
answers
185
views
multiple t test analysis or indidivual t tests
I have to analyse samples with t tests. I am not sure if I should treat my samples as individual t tests or multiple t tests. here the details:
I have an experiment where I analysed the concentration ...
- 51
-1
votes
1
answer
56
views
Usefulness of p-value to flag outliers in a data set [closed]
Suppose I have a set of data such that $$y= a\times x + b + \varepsilon $$
I am trying to find $a$ and $b$, but some $y$'s are outliers and up to 80% of the data is missing, so I don't have access to $...
- 1